Judges Flunk Story Problem Test, Showing Intuitive Decision-Making

Judiciary

Judges Flunk Story Problem Test, Showing Intuitive Decision-Making

Posted Feb 19, 2008, 10:20 am CDT
By Debra Cassens Weiss

Researchers who asked 295 Florida trial judges to solve three story problems have concluded their wrong answers help show judges rely primarily on intuition when making decisions.

Nearly a third of the judges failed to answer a single question correctly and a similar number got just one answer right. The judges who answered incorrectly tended to select the intuitively obvious, but inaccurate, responses. The results, although poor, were comparable to those of other well-educated adults.

The authors cited the test and other psychological studies of judicial decision-making in support of their conclusions. “Judges are predominantly intuitive decision makers, and intuitive judgments are often flawed,” says the article, called “Blinking on the Bench: How Judges Decide Cases.” The results of the study will be published in the Cornell Law Review, Legal Blog Watch reports.

The psychological studies show judges appear inclined to use intuition “when awarding damages, assessing liability based on statistical evidence, and predicting outcomes on appeal,” the article says. “They are also vulnerable to such distractions as absurd settlement demands, unrelated numeric caps, and vivid fact patterns.”

The article suggests that judges can overcome their intuitive tendencies if they are given more time to deliberate and they are encouraged to write opinions explaining their decisions. Training, peer review, and checklists could also be helpful.

The story problems are part of a so-called Cognitive Reflection Test designed to distinguish intuitive from deliberative processing. Here are the questions:

A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? (The answer is 5 cents, not 10 cents as many people first conclude.)
If it takes five machines five minutes to make five widgets, how long would it take 100 machines to make 100 widgets? (The answer is five minutes, not 100 minutes.)
In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake? (The correct answer is 47 days, not 24 days.)

The study's authors are Chris Guthrie of Vanderbilt University Law School, Jeffrey Rachlinski of Cornell Law School and U.S. Magistrate Judge Andrew Wistrich of the Central District of California.

Comments

Report Abuse

Posted by the trial guy - 7 months, 1 week, 6 days, 19 hours, 29 minutes ago

You mean judges actually got it wrong more than they got it right. Where’s the headline, the leading story on the nightly news. . . , oh that’s right this isnt news as practically every trial lawyer knows that many if not most trial judges are just as biased as the rest of the population whether they want to admit it or not. I’ll take the collective wisdom of 12 over the connected wisdom of one every time!!!
Posted by Odysseus Rex - 7 months, 1 week, 6 days, 18 hours, 21 minutes ago

SO WHO CARES about widgets??? What does this have to do with “who gets custody of a child”, or if joinder of a “real-party-in-interest” is appropriate? Now I remember why I quit the ABA many years ago. Hmmm...so what LOGICAL reason did I have to re-join?
Posted by Steven Jamar - 7 months, 1 week, 6 days, 14 hours, 22 minutes ago

These questions have nothing to do with making decisions. Few decisions are so simple and few have clear answers that can be derived mathematically.
While I would prefer judges who could solve these problems properly as well as have good common sense, I’ll take the judges who have common sense every day—especially at the trial level.
Posted by Michael - 7 months, 1 week, 6 days, 13 hours, 24 minutes ago

Judges fail math problems? No news here. An arithmetic-impaired lawyer myself, I can tell you that professionals go to their strength.

Now, put together a test that says “a 16 year-old girl wants an abortion, but her daddy objects, what do you do?”

Oh, and by the way, what IS the unambiguously correct answer to that?
Posted by Judge James Barlow - 7 months, 1 week, 6 days, 10 hours, 39 minutes ago

In each case someone wins and someone loses. The lawyers that win think the judge is very smart and the ones that lose think he is a dunce. Law professors are guys that do not have enough clients to practice and not enough moxie or political skills to become a judge. But they obviously think that if they were to make the rulings, then the rulings would be perfect.
So what-- that is what everyone else thinks also.
Posted by Mel Topf - 7 months, 1 week, 6 days, 10 hours, 8 minutes ago

The questions are simply trick questions, of the kind high shool kids like to have fun with They’re essentially designed to mislead. They tell nothing about a judge’s fairness, integrity, or prudence.
Posted by Alex - 7 months, 1 week, 6 days, 10 hours, 1 minute ago

I do not doubt the implications of this study. Judges are supposed to read the law, apply it to the facts and concude was the law broken or not. The study says that deductive mechanism is not happening.
This has major implications for the lawyer’s role. Our role is more art than science, and what we learn in law school or by reading the books is only marginally relevant to our client’s prospects.
Posted by Charles Montgomery - 7 months, 1 week, 6 days, 9 hours, 37 minutes ago

The most interesting part of the recommendation is that judges should Really explain how they weighed the key evidence and why they found certain evidence unpersuasive. Appellate courts often give trial judges a free pass on these requirements by allowing formulaic findings that explain little to the parties to substitue for a reasoned decision.
Thank you ABA for an interesting article.
Posted by Everett - 7 months, 1 week, 6 days, 7 hours, 13 minutes ago

There was nothing “tricky” about any of these questions: a potential answer, even a guess, can be immediately compared to a rather simple constraint. An incorrect answer shows a lack of reflection. This wonderful illustrates the importance of written opinions: element-by-element analysis highlights errors before they become holdings.
Posted by Got confused - 7 months, 1 week, 5 days, 18 hours, 8 minutes ago

Regarding problem 1, how can the cost of the ball be anything but 10 cents? someone plz explain
Posted by Michael Davis - 7 months, 1 week, 5 days, 13 hours, 35 minutes ago

If these were presented in an adversarial context, with an advocate for the irght answer and an advocate for the wrong answer, I think they would get these simole questions right, pretty much every time.
Posted by John Burgess - 7 months, 1 week, 5 days, 12 hours, 49 minutes ago

#11:

If the ball cost $0.10, then the bat would only cost $0.90 more than the ball. For the bat to be $1.00 more than the ball, the bat would have to sell for $1.05 and the ball for $0.05.
Posted by Mary - 7 months, 1 week, 5 days, 12 hours, 39 minutes ago

Well, duh! What do these questions have to do with being a judge?
Posted by Everett - 7 months, 1 week, 5 days, 11 hours, 16 minutes ago

In an adversarial situation, it is possible that neither side will be correct, as each lawyer pushes the envelope of plausible interpretations. That’s why holdings should be more than “defendant wins; no costs.” The act of writing a comprehensive decision explaining an opinion seems an excellent way to detect specious arguments eloquently advocated.
Posted by Scott - 7 months, 1 week, 4 days, 14 hours, 30 minutes ago

This is an idiotic study. The ability to solve mathematical word problems has, literally, nothing to do with understanding and parsing complicated fact patterns and creating cogent, cohesive and correct application of complex (non-linear) concepts to those fact patterns.

Utter stupidity --- especially since it starts with the dubious, and unstated, assumption, that all knowledge is mathematical in form, substance and source.
Posted by John - 7 months, 1 week, 4 days, 12 hours, 18 minutes ago

Looking at the many critical comments here, it is obvious that many readers are also “intuitive” thinkers, relying more on their visceral reactions rather than resorting to analytical or abstract thought.

Perhaps this is as it should be, and our conceit that the law is a rational probing process aimed at “justice” (as opposed to a gut-satisfying kind of fairness) is out of step with the demands and desires of the those that the legal system mostly serves.
Posted by Tony - 7 months, 1 week, 4 days, 9 hours, 17 minutes ago

It’s the judges whe aren’t intuitive thinkers that I have the problems with. You know, the ones that send the abused child back home because the rules say so, or the ones that split an estate using the mathematical axioms without considering the fact that only one of the kids helped care for daddy when he was alive. A little more intuition on the bench can go a long way. Instead, we keep seeing laws like 3 strikes and mandatory minimum sentencing guidelines that send petty criminals to already over crowded prisons for long stretches. I suppose its possible to eliminate judges and just have computers apply the rules. Then we could eliminate intuition altogether.
Posted by Mark - 7 months, 1 week, 3 days, 23 hours, 37 minutes ago

This ought to be on are you smarter than a fifth grader?

ps: I agree to the terms of service , but I don’t know what they mean. I’m a fourth grader.

Commenting has expired on this post.

ABA Journal - Law News Now

Judiciary