Subscribe Subscribe   Find us on Twitter Follow POL on Twitter  



Professor Saks responds, as do I

On April 3, I wrote a 560-word piece discussing the implication of some statistics cited in a Michael Saks op-ed, and why they were misleading. On April 11, Professor Saks asked for the opportunity to reply, and sent me a 2000+ word rebuttal on April 16, which is reprinted in full after the jump. I'm not going to address the condescending personal remarks. OK, I'll address just one: not that it's relevant, but, yes, I know who Richard Posner is, what with me being one of his former research assistants. I would like to discuss some of the substantive points briefly, though at more length than my original post. I don't doubt that Saks is already familiar with the literature I cite.

1. Saks can claim that I "do[] not challenge" his main conclusion that the malpractice crisis is caused by doctors only by ignoring everything I've written on the subject in the last two years. I don't believe the law of waiver applies when, while writing for a blog with regular readers, I fail to repeat every argument I've ever made when I write a post addressing a handful of particular details in a particular op-ed.

2. Saks's rebuttal and op-ed both rely heavily on the Harvard Medical Practice Study, a 1991 study of 1984 New York state hospital case records. The Harvard Medical Practice Study has been criticized elsewhere for substantially exaggerating the number and effect of medical errors, and I won't repeat those criticisms, but note that they eviscerate the premise of almost every argument Saks makes. Dr. Anderson's critique is worth reading on its own, and worth remembering next time you hear the claim that "medical errors cause 98,000 deaths." See also Rodney A. Hayward & Timothy P. Hofer, "Estimating Hospital Deaths Due to Medical Errors: Preventability Is in the Eye of the Reviewer," 286 JAMA 415 (2001); Clement J. McDonald et al., "Deaths Due to Medical Errors are Exaggerated in Institute of Medicine Report," 284 JAMA 93 (2000).

3. If Saks is going to rely on the Harvard Study, he has to take the bad with the good. The Harvard Study found that, holding severity of injury constant, the litigation system was just as likely to award damages in a case where no medical malpractice has taken place as one where medical malpractice has taken place; indeed, the sued non-negligent doctors paid more on average to injured patients than the sued negligent doctors, and the majority of patients receiving compensation weren't injured by negligence. See Brennan et al., "Relation between Negligent Adverse Events and the Outcomes of Medical-Malpractice Litigation," 335 NEJM 1963 (Dec. 26, 1996).

4. Furthermore, this omitted fact substantially weakens Saks's premise that the current malpractice litigation system serves as an effective deterrent. For litigation to be a deterrent, it has to have more effect on doctors who commit malpractice than those who do not; because of the utter randomness of litigation results, there's little evidence this is true in the current system. Mello and Brennan, "Deterrence of medical errors: theory and evidence for malpractice reform," 80 Tex. L. Rev. 1595 (2002). The only thing the current system provides is (1) a wealth transfer from doctors and patients to lawyers and (2) a random lottery effect for patients who have adverse results that may or may not be caused by malpractice. The main thing being deterred is the actual practice of medicine.

5. Even while ignoring this finding of the Harvard Study, Saks unironically cites numbers from that study that say that 83% of malpractice suits filed are meritless. I would say that this supports my claim that lawyers do a poor job of screening meritless cases; Saks argues that this refutes it. I'm confident in agreeing to disagree here over the meaning of "poor job." Regardless, I stand by my criticism that the "29 times more likely" statistic is a misleading way to communicate how well the tort system screens cases: this sort of false positive conjecture is criticized as innumeracy when journalists do it, and it's not unreasonable to hold a professor to a higher standard. I picked round numbers to illustrate the basic statistical principle that "29 times more likely" was not an impressive statistic when one was talking about something as rare as medical malpractice. (Professor Volokh did something similar two weeks later in talking about rape accusations.) Professor Saks criticizes at length the fact that the plainly hypothetical example "ma[de] it all up out of thin air," but fails to explain why it makes a jot of difference to my argument that I used obviously fictional numbers rather than fictional numbers masquerading as definitive ones.

6. Speaking of ignoring findings from the Harvard Study, it's an entertaining irony to note that the Harvard Study recommended placing limits on pain-and-suffering damages, the very proposal Saks fulminates against in his op-ed.

7. Saks's original op-ed said that doctors had no right to complain about high malpractice insurance costs because they could pass those costs along to their patients. I noted that this contradicts his claims that malpractice litigation serves as a deterrent. Saks simply repeats the contradiction. He can't have it both ways: either doctors suffer from the expense of malpractice insurance, or they do not, and if they do not, then their indifference means that there is no deterrent effect from higher malpractice insurance rates. As Saks finally acknowledges in his point #2, higher malpractice insurance rates reduce doctors' profits. Doctors thus have reason to complain about the high malpractice insurance rates caused by the inefficient litigation system, and Saks's op-ed statement to the contrary is too strong—which was exactly the point I made in my original post. His response defends a different argument than the one he made.

8. I do not "question[] the finding that a proportionately small number of doctors is responsible for a proportionately large number of injuries, suits and damages." I question only the "finding" with respect to injuries, for which Saks cites no evidence (though it wouldn't surprise me if it were trivially true: a brain surgeon is more likely to cause an injury than a dermatologist). With respect to suits and damages, I don't disagree with the finding, but I question the relevance of that finding, given the randomness of the litigation system and the different risk profiles of different practices. I don't understand why Saks is throwing the Gittler-Bovbjerg study at me; my original post linked to my Jan. 6 discussion of that study, which he doesn't address, and which I won't repeat here. Once again, Saks is talking past my criticism.

9. I questioned Saks's claims that "research finds that only 10 to 20 cents are paid on each dollar of economic loss." In response, Saks points me to Frank Sloan's work. But in Sloan's Suing for Medical Malpractice, Sloan found that plaintiffs who went to trial in his study set of malpractice cases, in the words of Neil Vidmar, "ultimately receiv[ed] 22 percent more than their estimated economic losses." I would further submit that Sloan's analysis overestimates economic losses caused by malpractice, such that the real percentage is actually higher, but that's an issue for another day.

Professor Saks writes:

On Ted Frank's Series of Unfortunate Errors,
Or, The Risks of Making it All up Out of Thin Air

Michael J. Saks

(Reply to Ted Frank's Blog of April 03, 2005,
titled "How to lie with statistics, medical malpractice edition"

Ted Frank takes issue with an op ed piece I wrote on proposals now before Congress to effect a federal takeover of some state law responses to injuries to patients caused by avoidable medical errors. I'm accustomed to plenty of disagreement and debate; I have far less experience with the kind of ad hominem attacks that Mr. Frank feels the need to rely on. Being knowledgeable about the facts, but far less adept at the ad hominems, I'm going to deal with the facts about medical error, the law's responses, and inferences from those facts, and leave it at that.

I will begin, however, by pointing out that we agree on matters of central importance. Frank does not challenge my statements that: "The best way to prevent medical malpractice litigation is to reduce the incidence of medical errors and injuries. Studies conducted over the past 30 years have repeatedly found them to be the leading cause of accidental death and injury in the U.S., exceeding all other causes combined. The President's solution to this public health catastrophe is to protect insurers and injurers at the expense of seriously injured patients." Those are the main points. Everything else is details. And on those details Frank gets it wrong at every step.

1. Frank questions my statement that doctors pay for the cost of malpractice premiums out of the fees they collect for their services, and therefore that such costs ultimately are paid by consumers. When required to compensate for harm they have caused, all businesses either pay directly from their revenues (they self-insure) or they purchase insurance to cover those costs which might arise (they purchase liability insurance). As every student of law and economics knows, this is in essence how the compensation aspect of tort law works. (See, for example, the work of Chicago's Posner or Yale's Calabresi or others in the national army of economically literate legal thinkers.) The purpose is not to hurt businesses or professionals. It is to make the compensation of victims injured by the negligent acts of others part of the cost of doing business by internalizing the costs from the injured to the injurers. The money to pay those costs is spread among the consumers of the particular business. The business must add something to the price it charges each consumer so that it has available the money to compensate those whom it negligently harms.

2. Frank thinks that if the above is true, it is a contradiction to say that damages serve as a deterrent. After all, if businesses pass the cost along to the consumer, where is the deterrent? But the second prong of the elementary law and economics of tort law speaks to the deterrent effect of damages. In the classical situation of the self-insuring business, a safer business needs to pay out less in damages, and therefore can charge less than its more careless competitors, and its lower prices attract more customers, and it makes more profit. Or it keeps the money it set aside to pay for injuries as a windfall profit. For less safe businesses and professionals, the exact opposite obtains: higher prices, less business, less profit. With the advent of insurance and managed care, the situation grows more complicated, but deterrence still does not disappear. Economic analysis (by Georgetown's Zeiler) has found that as malpractice premiums are reduced artificially (by caps, rather than as a consequence of safety measures) managed health care organizations adopt policies and practices that tolerate greater risk to patients. As damages go up, insurers and professional groups work harder to make their members act more safely. And, without the protection of artificial ceilings on damages, individual doctors are deterred by the fact that a damage award might exceed the limits of their malpractice policies. They will pay the cost of the harm they do and not be assured that they can pay something less.

3. Frank questions the finding that a proportionately small number of doctors is responsible for a proportionately large number of injuries, suits and damages. The most recent confirmation of this finding comes from a study commissioned by the Bush administration and released earlier this year (by the University of Iowa's Gittler and the Urban Institute's Bovbjerg). (See the New York Times, Robert Pear, "Panel Seeks Better Disciplining of Doctors," January 5, 2005.) Bovbjerg is quoted: "If you take the worst performers out of practice, that will have an impact [on malpractice litigation]. Most doctors have few or no claims filed against them. But within any specialty, a few doctors have a high proportion of the claims." If one has a sense of the bell curve distribution of performance of people in most fields (a small proportion of stars, many in the middle, and a small proportion of incompetents), then these findings will seem fairly commonsensical. Only a small percentage of doctors are addicts, alcoholics, mentally ill, or otherwise disturbed and dangerous. But the medical profession has been unusually successful at preventing boards of licensure and discipline, as well as private insurers (by paying extra high premiums in exchange for no experience rating of the insureds), from weeding out any members of the guild, including many of the worst performers � at the expense of both other doctors and the public.

4. Frank guesses that "doctors commit malpractice on 1 in 200 patients" � a figure he thinks "seems awfully high." Even his low estimate, which he thinks is a high estimate, is way too low. Findings from different empirical studies, by different entities (medical organizations, academic, and governmental) have found between .79 and .93 per 100 patients � nearly twice as many as Frank's guestimate. These findings come from intensive reviews of hospital medical records. Other studies have found that doctors succeed in hiding critical facts about many injuries from the medical records, so that the true rate of negligent injuries is between double and quadruple the amount found from the records reviews quoted above. That would put the rate of medical malpractice (for hospital inpatients) more realistically somewhere between 4 and 8 in 200 (to use Frank's denominator).

5. Relatedly, in insisting that the tort system is bad at screening out cases that lack merit, Frank rejects or is ignorant about numerous research findings, replacing them with his own speculations. But this is a slightly complicated statistical matter, and I would have some sympathy for Frank's confusion even if he were trying to deal with real facts instead of his own speculations. But I think the real (if slightly complex) picture can be understood. Consider the following table of data from the Harvard Medical Practice Study of New York state doctors and hospitals (perhaps the single best study of the question of how much malpractice there is and how much of it turns into claims for compensation).

Claim Filed?
Negligent Adverse Event2728280
No Neg Adverse Event29,8023929,841

First, note that the 280 cases of what the medical reviewers concluded were negligent medical injuries (that is, malpractice) represented nearly one percent of the 30,121 patients whose records were studied.

Second, of the 280 cases of medical malpractice, only 8 (or 2.9%) resulted in filed claims for compensation. The remaining 97% of patients injured by malpractice did not seek the compensation to which they would have been entitled. Thus, even doctors who commit malpractice are extremely unlikely to be sued (and they know it � judging from studies which asked them to guestimate these probabilities).

Third, of the patients who suffered no injuries, or who suffered injuries judged not to have been caused by negligence (29,841 of them), 39 (or one-tenth of 1%) nevertheless filed claims. (By the way, few of these claims could be termed frivolous, even though they ultimately were judged by the researchers not to involve negligence. In many instances, at least one of the two medical records reviewers thought the record disclosed malpractice, but were overruled by a tie-breaking researcher/reviewer. The lesson: it can be hard to know when a medical injury is due to negligence and when it is not.)

Fourth, if you form a ratio of the proportion of claims brought by negligently injured patients (.029) and the proportion of claims brought by non-negligently injured patients (.001) you get 29. This is the figure that Frank calls "the most jarring statistic," and which he calls "bogus." But, like it or not, that is what the data show to be the relative risk of being sued by a patient who has been a victim of malpractice compared to being sued by a patient who has not been a victim of malpractice. Note, however, that more interesting than the fact that a doctor is 29 times as likely to be sued by a patient who has been the victim of malpractice than by one who has been the victim of non-negligence or bad luck is the fact that the probability of being sued even by a patient who has been a victim of malpractice is only .029.

But, fifth, if you look at the 47 filed claims, only 8 (17%) of them were injured by malpractice and are legally meritorious cases. Conversely, 83% of filed claims lacked merit. On this Frank stumbled fairly close to the mark. (I say stumbled because he used incorrect input data and wrong assumptions, but still landed close to the right answer. On the other hand, he need not have guessed. There are hard data and they've been known for decades.)

But, importantly, Frank completely misunderstands the meaning of this figure. It does, indeed, appear scary to a malpractice insurer: only one of every five filed claims is truly meritorious. But consider the full picture. Who is well served by this "inaccurate" system? If you were an insurer or a premium payer, would you prefer an "accurate" system? Would you prefer to defend suits brought by all 280 of the negligently injured or by the 47 claims that were filed, four-fifth of which are relatively weak cases?

And why are there relatively so many cases being brought for injuries not caused by malpractice? Frank gets that wrong, too. One reason, as mentioned above, is that it can be hard to distinguish injuries caused by negligence from injuries caused by something else. (Medically trained records reviewers with no axes to grind don't always have an easy time telling them apart.)

But there also is an inescapable mathematical reason understood by every statistician and every doctor who was paying attention in medical school: when you have a skewed baserate of whatever it is you are trying to detect, even a highly accurate test will produce a large number of false positives. If you are screening a population of children for a disease or checking luggage for bombs, even a test that is 99.9% accurate will produce many more false positives than true positives.

The baserate of negligent medical injuries is highly skewed: one negligent injury for every 107 non-injuries or non-negligent injuries. The data on medical malpractice imply that, even at an early stage of the process (filing), the system is 99.9% accurate at keeping true negatives out. But that tiny error rate allows (out of 30,000 cases) 39 false negatives to get filed. Conversely, on the true positive side of the coin, the system is highly inaccurate in that the vast majority of the truly negligently injured patients do not file. People who have studied the data know, and have known since the earliest of these studies in the 1970s, that this is a system that is very slow to act for victims and very protective of potential defendants.

6. Finally, Frank doesn't know the data on the compensation of meritorious medical malpractice claims (whether by settlements or trial judgments). He even Googled it, but still couldn't find it, and calls out to readers for help. The empirical literature on tort compensation is quite large, and yet one's scholarship might have to extend beyond Google. I was quoting from research by the Duke economist Sloan. But one might have looked at work on tort damages by a hundred, or a thousand, other researchers. The fact is that under-compensation, even of special damages, is the rule, and over-compensation (except for the smallest injuries) is the exception.



Isaac Gorodetski
Project Manager,
Center for Legal Policy at the
Manhattan Institute

Katherine Lazarski
Press Officer,
Manhattan Institute


Published by the Manhattan Institute

The Manhattan Insitute's Center for Legal Policy.