AI research is useful only after the lawyer verifies it
A new comparison study should end the sales pitch that legal AI replaces research instead of sitting in front of it.
An AI-assisted editorial, reviewed by a human before publishing. It reasons over our own tracker data (and cited context). A point of view, not legal advice.
Westlaw’s AI-Assisted Research got more than 34% of its answers wrong or misgrounded, and Lexis+ AI and Ask Practical Law AI were wrong or misgrounded in more than 17% of queries. Those are not edge cases. They are the kind of numbers that should make any lawyer stop pretending the tool has crossed the line from assistant to substitute.
The immediate temptation, of course, is to treat this as a product race. Anthropic has appointed Robert Mahari as its first official Head of Claude for Legal, and Microsoft’s legal and compliance arm is already using Harvey. LexisNexis is partnering on in-house workflows inside Luminance, and Filevine says it is shipping AI into real legal work with evaluation-driven development. The market is not waiting for certainty. It is buying seats.
The only honest product promise left is verified workflow.
The old lesson was never about novelty
The profession has already seen what happens when a lawyer trusts generated text before checking the source. In a 2023 Southern District of New York matter, a lawyer was challenged over seven apparently fictitious cases, asked ChatGPT to summarize them, and the court imposed a $5,000 fine on the two lawyers and their firm. Another 2023 sanction matter ended the same way: two lawyers filed a brief with fake cases after one used ChatGPT, and each lawyer plus the firm was fined $5,000. In Harber v Commissioners for His Majesty’s Revenue and Customs, a litigant in person filed an appeal submission citing nine non-genuine authorities hallucinated by generative AI. The machine was not the problem. The failure to verify it was.
That was the first lesson, and too many people seem determined to relearn it in a nicer interface. The new comparison study matters because it takes the issue out of disciplinary horror stories and puts it where procurement lives: if a research tool is misgrounded often enough, then the question is no longer whether the output sounds fluent. The question is whether anyone should buy it as a substitute for the thing lawyers have always been paid to do, which is find the law and check it against the sources that actually count.
This is now a workflow problem, not a demo problem
The study’s answer is annoyingly plain. The real shift is hybrid: use AI to map and synthesize, then verify with Westlaw or Lexis citators and editorial sources. That is a workflow, not a replacement. It is also the only sensible answer if you care about ethics, because a tool that can be wrong in more than one out of every six queries, and worse than that in Westlaw’s own AI-assisted product, does not deserve to sit alone in front of a client matter.
The context makes the point sharper. According to the ABA Journal, Am Law 100 firms said they were spending more on artificial intelligence and innovation than on any other expense category, and nearly 70% of respondents reported AI and innovation investments had increased more than 5% through the second quarter of the year. That is not a pilot project on the side. That is institutional buying. When firms are spending real money and in-house teams are embedding tools into actual practice, accuracy stops being a theoretical concern and becomes a vendor-selection issue, a training issue, and an ethics issue all at once.
What lawyers should stop buying
The profession needs to retire the fantasy that legal AI is a replacement for research rather than a front end for it. Claude for Legal, CoCounsel, Harvey, and Legora all have obvious utility. CoCounsel can summarize documents, draft deposition outlines, review documents, and generate legal research memos. Harvey offers agents, knowledge, vault, contract intelligence, and workflow tools. Legora helps with drafting, research, and document review. None of that changes the basic fact that the lawyer still has to know when the answer is grounded and when it is just tidy prose in a suit.
Vendors can keep selling speed. Lawyers should insist on verification. That is the real procurement standard now: not whether the tool can produce something that looks like legal work, but whether the workflow forces the lawyer back to the database, the citator, and the editorial source before anything leaves the firm. The market already knows how to package confidence. It needs to get a lot better at packaging caution.
The bill is arriving where the adoption is
There is a reason this matters right now rather than in some abstract future. Legal AI tools are being institutionalized inside firms and in-house teams while the failure rate remains visible. The site tracks 215 legal AI tools, and the ones getting the most attention are Claude for Legal, CoCounsel, and Harvey. Attention is not quality, and it certainly is not trust. Yet attention is what usually gets budgets approved, and budgets are what turn experiments into habit.
The lesson from the archive is still the lesson: check the source, or pay for the mistake. The only honest product promise left is verified workflow, because a tool that can talk like a researcher is still not a database.