Subscribe Subscribe   Find us on Twitter Follow POL on Twitter  



"Insightful or Wishful: Lawyers' Ability to Predict Case Outcomes."

Fascinating paper by Goodman-Delahunty et al. (via ABA Journal):

Lawyers' litigation forecasts play an integral role in the justice system. In the course of litigation, lawyers constantly make strategic decisions and/or advise their clients on the basis of their perceptions and predictions of case outcomes. The study investigated the realism in predictions by a sample of attorneys (n = 481) across the United States who specified a minimum goal to achieve in a case set for trial. They estimated their chances of meeting this goal by providing a confidence estimate. After the cases were resolved, case outcomes were compared with the predictions. Overall, lawyers were overconfident in their predictions, and calibration did not increase with years of legal experience. Female lawyers were slightly better calibrated than their male counterparts and showed evidence of less overconfidence. In an attempt to reduce overconfidence, some lawyers were asked to generate reasons why they might not achieve their stated goals. This manipulation did not improve calibration.

Unfortunately, because the paper focuses purely on trial outcomes, it is less useful than one would hope. In a world where the vast majority of cases settle, a case will not go to trial unless one or both parties are overconfident and, because of this overconfidence, fail to agree to a reasonable settlement offer. (If a party is underconfident, then it will be more likely to settle, and settle for too cheaply; if both parties have realistic views of the likelihood of success, it will always be cheaper to settle than incur litigation costs. An underconfident attorney will go to trial only if the opposing attorney is more overconfident than the underconfident attorney is underconfident.) Thus, because the study restricts itself to cases where there was a trial, it biases its outcomes. The study fails to cite any of the law-and-economics literature on the subject.

However, I did find fascinating Figure 2 on page 142. It seems to show that there's an issue of underconfidence as well: attorneys who thought their clients had less than a 45% chance of success (mean about 27%) won nearly half the time, while attorneys who thought their clients had more than a 55% chance of success won slightly more than half the time. This would suggest that either lawyers who go to trial do a poor job of assessing trial probabilities—or that trials are very close to a coin-flip.

Disclosure: I worked with one of the co-authors, Elizabeth Loftus, who served as an expert witness on a trial where I represented the defendant; she did not testify because it settled shortly after opening statements.



Rafael Mangual
Project Manager,
Legal Policy

Manhattan Institute


Published by the Manhattan Institute

The Manhattan Insitute's Center for Legal Policy.