Marc Rodwin's response to my critique of his lead-authored study for Health Affairs consists entirely of statements that are either (1) factually false or (2) irrelevant to the central criticism I made of the paper, which is never addressed: Rodwin's bottom-line conclusions are cherry-picked non sequiturs from the underlying data in the paper.
What does it mean to say that Massachusetts is a state in medical malpractice crisis? The AMA does not define its terms, but there are several possible definitions:
1a) Rising insurance rates over the long term.
1b) Rapidly rising insurance rates in the immediate short term.
2a) Rising insurance rates for "high-risk" specialties like neurology or obstetrics over the long term.
2b) Rising insurance rates for "high-risk" specialties like neurology or obstetrics in the immediate short term.
3) Doctors shifting to lower-risk practices because of insurance expense.
4) Doctors leaving practice because of insurance expense.
What did the Rodwin study find? Let's look at Exhibit 1. Unfortunately, Exhibit 1 of the Rodwin paper would be more appropriate for Exhibit 1 in a Edward Tufte book about how not to display data. Rather than present data in a readable format, it is obfuscated with thick spotted lines and data points. At no point in the paper does Rodwin present actual five-digit numbers from 1975, 1980, or 1985, despite the "1975-2005" in the title. But we can make rough estimates from what Rodwin does supply.
1a) Using the beginning and end points of the study, the most natural starting and end places for comparison, the mean insurance rate that a doctor paid in Massachusetts more than doubled (after inflation).
1b) Using the most recent five-year interval, 2000-2005, the second-most natural comparison point for recent trends (given the study only measured seven time-slices, 1975, 1980, 1985, 1990, 1995, 2000, and 2005), the mean insurance rate that a doctor paid in Massachusetts went up between 25% and 40% (after inflation). Again, the exact percentage is unavailable because the data is obfuscated.
2) Restricting the study to just high-risk doctors, ("Tier 1" by Rodwin's classification), we see that the mean insurance rate that a doctor paid in Massachusetts went up from about 1975's $20,000 (in 2005 dollars) (and, again, an estimate because of data obfuscation) to $95,045 (actual figure from Rodwin), an increase between 300% to 400% after inflation. The 2000 figure was $65,612, a 45% increase.
3) Rodwin releases no data for 1975, 1980, or 1985. But in 1990, 8% of doctors had Tier-1 policies. In 1995, about 5% of doctors did. In 2000, about 4% of doctors did. In 2005, fewer than 4% of doctors did. (Exhibit 2.) The number of doctors in the top 4 tiers went from about 27% to 20%. Not once in the paper does Rodwin mention the shift in risk classifications, but the data shows that a different mix of doctors was buying insurance in 2005 than in 1990.
4) Rodwin does not measure the number of doctors practicing.
So by three out of four criteria for saying that a state is in crisis, Rodwin's data shows Massachusetts in crisis, and doesn't measure the fourth. But Rodwin concludes flat-out that Massachusetts is not in crisis, because he measures "crisis" by a single slice: 1990 premiums vs. 2005 manual premiums for a $1M/$3M policy—even though his own data shows that the mix of doctors has changed between 1990 and 2005, and that the reason for the rate decrease is Simpson's Paradox. When one makes the apples-to-apples comparison of Tier 1 rates, the most relevant for discussing "crisis," one finds a tremendous 45% increase for $1M/$3M policies. (Tier 5 is up 14%—more if one accounts for doctors switching to $2M/$6M policies; Tier 4 is up slightly; Tiers 2 and 3, accounting for about 6-7% of doctors, is down.) Simpson's Paradox is not mentioned in the paper. (And even between 1990 and 2005, the mean insurance rate went up if one accounts for doctors purchasing more coverage because the nominal $1M/$3M policy provided less protection in 2005 than it did in 1990.)
At best, Rodwin can make the arguable conclusion "The malpractice crisis is no worse than it was in 1990." I'd disagree with that because of Simpson's Paradox, but it's at least arguable. Rodwin goes further, and flatly says there is no malpractice crisis. His data does not support that. And if he picked any other starting data point, he wouldn't even be able to use Simpson's Paradox to say that rates went down. And at no point in his response does Rodwin explain why he chose 1990 as the reference point.
Let's correct the false statements in Rodwin's response:
- Rodwin: Contrary to what Ted Frank's posting states, the Massachusetts data found that mean premium rates for all physicians increased only slightly from 1975 to 2005.
That is obviously false. See Exhibit 1 of the Rodwin paper, which plainly shows otherwise, even after data obfuscation.
- Ted Frank makes the usual misleading point by discussing premium increases in terms of percentage increases, rather than dollar increases.
Also false. I discuss dollar-amounts in the post almost as often as percentages, and, in any event, link back to the original paper for others to check my conclusions. I'd be happy to discuss the tremendous dollar increases in Exhibit 4, which confirm the medical malpractice crisis for spinal surgeons ($27,000 [sic] increase from 2000-2005, inflation adjusted), neurologists ($27,575-$48,438), and obstetricians ($32,807-$36,239).
- Ted Frank fails to acknowledge any premium decreases over 30 years. ... Ted Frank, in his own cherry picking points out premium rises, ignores their declines, and does not indicate long term trends.
False. To quote from my post, "Mean 2005 malpractice insurance rates are up astronomically since 1975 and 1980 (all numbers adjusted for CPI inflation) and up substantially since 1995 and 2000. (They are up slightly since 1985, and less than 1% lower than in 1990.)" I then repeat "Mean rates have gone down slightly since 1990." It is Rodwin who fails to note the long-term 1975-2005 trend. And at no point does Rodwin defend his indefensible use of a 1990 starting point.
- In 1990, premium rates for 72 percent of physicians were less than $20,000 while in 2005, premium rates for 78 percent of physicians were less than $20,000 a year for the most typically purchased type of policy.
Misleading, since it fails to account for the change in the mix of physician insurance. The $20,000 figure (and 1990 start-date) is cherry-picked to be the most favorable to Rodwin's argument. And largely irrelevant, since the critical issue is obstetricians, ER doctors, and other high-risk specialties.
- Contrary to what Ted Frank states, we don't mix inflation adjusted data and non-inflation adjusted data. All our data is expressed in 2005 dollars.
False. See Exhibit 2, Exhibit 3, Exhibit 4, and Exhibit 5, where insurance policy limits are expressed in nominal dollars while premiums are expressed in 2005 dollars. Indeed, the only reason that Rodwin can say that insurance rates went down from 1990 to 2005 is because he uses inflation-adjusted data to measure the cost of the non-inflation-adjusted value of the policy. If he had used nominal dollars for both, we'd see that rates had gone up substantially; if he had used inflation-adjusted dollars for both, we'd see that the same dollars are buying less coverage.
- The study compares the premiums cost for the same policy type and coverage level. We reveal that physicians have purchased policies with different coverage levels, but these changes are not as Ted Frank states. There have been some increase in dollar amount of insurance purchase and some increased in purchase of less expensive policy types. The changes are analyzed and detailed in the text and exhibits.
Dodges the argument I'm making. There is some discussion about the change of mix of policies on pages 840 and 841, but at no point is Simpson's Paradox addressed, and at no point does Rodwin address the implications of the change of mix of policies in understanding why mean rates declined from 1990 to 2005.
- Unlike Ted Frank, who makes unsubstantiated claims about physicians being forced from the market by high premiums, we report only real data. We know of no other published piece that provides such authoritative data.
Rodwin's own data shows a substantial decrease in Tier-1 doctors purchasing insurance—a decrease large enough to cause the rare Simpson's Paradox. Again, at no point does Rodwin address the implications to his data of the changing mix of doctors as high-risk doctors purchasing expensive insurance leave the data pool, much less the implications to the claims of crisis. Numerous empirical studies reflect that the litigation environment will affect the supply of doctors at the margin, especially in high-risk specialties: Klick and Stratmann (2007); Klick and Stratmann (2005); Kessler, Sage, and Becker (2005); Matsa (2005); Dranove and Gron (2005); Dubay, Kaestner, and Waidmann (2001); Mello et al (2005).
Rodwin spins his wheels with a lot of words trumpeting the quality of his data collection, but this misses the point: I never say anything about the data collection. My criticism is of his data analysis, and of the conclusion not supported by the data. At no point in his response does Rodwin defend his reasoning or his conclusion. There are two papers in the Rodwin paper: empirical data at pages 835-842, and a series of personal opinions unsupported or contradicted by the data at pages 842-844. The press release then highlights the opinions and falsely implies they are empirical findings.