ABA Journal

The New Normal

What if Someone Could Measure What Lawyers Do?


By Paul Lippe

  •  
  •  
  •  
  •  
  • Print.

Based on what we’re seeing with sophisticated legal departments, the four horsemen of the New Normal will be:

• Legal process outsourcing.
• Substitution of technology for people in repetitive work.
• Treating legal work as teamwork rather than individual work.
• The emergence of one or several standards for measuring quality (perhaps value) in legal services.

The New Normal doesn’t claim that 100 percent of today’s legal work gets disrupted in this way, but 30 to 40 percent will—with significant impact.

Yet whenever we introduce this topic for the first time, some skeptic will insist these things are impossible. Nowhere is that more true than when we talk about measuring lawyer quality, which invariably yields: “Well, of course you can’t measure what I do.”

The fancy term for that harrumph is that law is a “credence good,” i.e., one where the seller is in a much better position than the buyer to determine the type and value of service provided.

Let’s begin by stipulating that within the mix of legal services, some are harder to measure than others. But even so, most legal services are far more measurable than most practitioners would recognize, and most attempts to measure quality, even if imperfect, will improve performance. The assertion that you can never reduce costs without hurting quality is clearly false, and undermines the credibility of the profession. Most of the sensible things that we can do to reduce costs will improve quality as well.

In other credence-good fields, such as medicine and academia, there have been systematic attempts over the last 30 years to begin to measure performance. Medicine is more advanced, and the model most of us know is the 73 year-old standard for Food and Drug Administration approval of the safety, efficacy and cost-effectiveness of drugs, based on the scientific method. For its part, academia has been as resistant to change as law, and the result has been the emergence of U.S. News rankings as a very flawed but pervasive way of comparing universities.

In all fields, the conversation tends to start with a focus on costs, which leads to the rejoinder that if you question costs, you put quality at risk, which, after a few twists and turns, will hopefully lead to a serious discussion of what quality actually is.

While there are many analogues to medicine that can be applied to law (as I hope to discuss in a subsequent post), it seems unlikely that one would ever be able to run the direct equivalent of a “blind study” comparing a placebo with a specific intervention. But what we can do is compare any legal matter to a similar matter for the same client, which could be either:

• Many like matters (e.g., sales contracts) for the same client.
• Matters delivered over time by different firms for the same client.
• Matters delivered by firms versus matters delivered by alternative service providers like legal process outsourcers.
• Matters delivered by the legal department with little or no firm involvement.

Once you acknowledge there can be some basis for comparison, you then want to think about what criteria you would like to assess, even as we recognize that data collection in law will be imperfect. Using the sales contracting example, the criteria might be:

• How quickly did the contract get done?
• How favorable are the terms to the company (opportunity gained and risk avoided)?
• How easy are the terms for other parts of the company (finance, manufacturing, sales, etc.) to understand and perform?
• How satisfied were the true business clients?
• How satisfied was the counterparty?
• How much did the contract cost?
• Did the contracting process improve?

In all likelihood, the company’s quality criteria would be different from most lawyers’ (who tend to focus on the comprehensiveness of the contract and its one-sidedness), illustrating that just the exercise of being explicit about quality is beneficial.

Without doubt, many of the elements of legal quality may not become evident for a long time frame. But more importantly, all the research on performance in organizations now shows that focused planning (which requires information sharing) and systematic feedback leads to higher performance.

Last week, I had the pleasure of speaking to Tanina Rostain’s Technology, Innovation and Legal Practice class at Georgetown Law School. When I suggested that law would begin to follow medicine in the attempt to systematically measure quality, the students’ initial reaction was: “That couldn’t work, medicine is too different.”

But as we worked through a scenario, the class was quickly able to see that it would be both possible and useful to try to rigorously measure quality in certain domains in law.

In the wake of the Facebook IPO, it is little short of madness not to recognize that transparency is one of the driving forces of our time. Lawyers who embrace the opportunity to measure and improve their quality will win; those who (with little or no empirical support) continue to claim that quality is only in the eye of the provider will do themselves and their clients no good.


Paul Lippe is the CEO of the Legal OnRamp, a Silicon Valley-based initiative founded in cooperation with Cisco Systems to improve legal quality and efficiency through collaboration, automation and process re-engineering.

In This Podcast:

Give us feedback, share a story tip or update, or report an error.