Oct 28, 2025

A new benchmark study released by Vals AI suggests that both legal-specific and general large language models are now capable of performing legal research tasks with a level of accuracy equaling or exceeding that of human lawyers. The report, VLAIR – Legal Research , extends the earlier Vals Legal AI Report (VLAIR) from February 2025 to include an in-depth examination of how various AI products handle traditional legal research questions. That earlier report evaluated AI tools from four vendors — Harvey, Thomson Reuters (CoCounsel), vLex (Vincent AI), and Vecflow (Oliver) — on tasks including document extraction, document Q&A, summarization, redlining, transcript analysis, chronology generation, and EDGAR research. This follow-up study compared three legal AI systems – Alexi, Counsel Stack and Midpage – and one foundational model, ChatGPT, against a lawyer baseline representing traditional manual research. All four AI products, including ChatGPT, scored within four points of each other, with the legal AI products performing better overall than the generalist product, and with all performing better than the lawyer baseline. The highest performer across all criteria was Counsel Stack. Leading Vendors Did Not Participate Unfortunately, the benchmarking did not include the three largest AI legal research platforms: Thomson Reuters, LexisNexis and vLex. According to a spokespeople for Thomson Reuters and LexisNexis, neither company opted into participating in the study. They did not not say why. vLex, however, originally agreed to have its Vincent AI participate in the study, but then withdrew before the final results were published. A spokesperson for vLex, which was acquired by Clio in June , said that it chose not to participate in the legal research benchmark because it was not designed for enterprise AI tools. The spokesperson said vLex would be open to joining future studies that fit its focus. Overview of the Study Vals AI designed the Legal AI Report to assess AI tools on a lawyer-comparable benchmark, evaluating performance across three weighted criteria: Accuracy (50% weight) – whether the AI produced a substantively correct answer. Authoritativeness (40%) – whether the response cited reliable, relevant, and authoritative sources. Appropriateness (10%) – whether the answer was well-structured and could be readily shared with a client or colleague. Each AI product and the lawyer baseline answered 210 questions spanning nine legal research types, from confirming statutory definitions to producing 50-state surveys. Key Findings AI Now Matches or Beats Lawyers in Accuracy Across all questions, the AI systems scored within four percentage points of one another and an average of seven points above the lawyer baseline. Lawyers averaged 71% accuracy. ChatGPT: 80%. When grouped, both legal-specific and generalist AIs achieved the same overall accuracy of 80%, outperforming lawyers by nine points. Significantly, for five of the question types, on average, the generalist AI product provided a more accurate response than the legal AI products, and one question type where the accuracy was scored the same. “Both legal AI and generalist AI can produce highly accurate answers to legal research questions,” the report concludes. Even so, the report found multiple instances where the legal AI products were unable to produce a response. This was due to either technical issues or deemed lack of available source data. “Pure technical issues only arose with Counsel Stack (4) and Midpage (3), where no response was provided at all. In other cases, the AI products acknowledged they were unable to locate the right documents to provide a response but still provided some form of response or explanation as to why the available sources did not support their ability to provide an answer.” Legal AI Leads in Authoritativeness While ChatGPT matched its legal-AI rivals on accuracy, it lagged in authority — scoring 70% to the legal AIs’ 76% average. The difference, Vals AI said, reflects access to proprietary legal databases and curated citation sources, which remain differentiators for legal-domain systems. “The study outcomes support a common assumption that access to proprietary databases, even if composed mainly of publicly available data, does result in differentiated products.” Jurisdictional Complexity Remains Hard for All All systems struggled with multi-jurisdictional questions, which required synthesizing laws from multiple states. Performance dropped by 11 points on average compared to single-state questions. Counsel Stack and Alexi tied for best performance on these, while ChatGPT trailed closely. AI Excels at Certain Tasks Beyond Human Speed The AI products outperformed the lawyer baseline on 15 of 21 question types — often by wide margins when tasks required summarizing holdings, identifying relevant statutes, or sourcing recent caselaw. For example, AI responses were completed in seconds or minutes, compared to lawyers’ average 1,400-second response latency (~23 minutes). And where the AI products outperformed the humans on individual questions, they did so by a wide margin – an average of 31 percentage points. Human Judgment Still Matters Lawyers outperformed AI in roughly one-third of question categories, particularly those requiring deep interpretive analysis or nuanced reasoning, such as distinguishing similar precedents or reconciling conflicting authorities. These areas underscore, as the report put it, “the enduring edge of human judgment in complex, multi-jurisdictional reasoning.” Methodology The study was conducted blind and independently evaluated by a consortium of law firms and academics. Each participant answered identical research questions crafted to mirror real-world lawyer tasks. Evaluators graded every response using a detailed rubric (which the report includes). The AI vendors represented were: Alexi – legal research automation startup (founded 2017). Counsel Stack – open-source legal knowledge platform. Midpage – AI research and brief-generation tool. ChatGPT – generalist large language model (GPT-4). Vals AI cautioned that the benchmark covers general legal research only, not tasks such as drafting pleadings or generating formatted citations. And, as the report notes, “Legal research encompasses a wide range of activities … but there is not always a single correct answer prepared in advance.” Bottom Line The VLAIR – Legal Research study reinforces what many in the legal tech industry have already observed, which is that AI systems – both generalist and domain-trained – are rapidly closing the quality gap with human legal researchers, particularly in accuracy and efficiency. Yet, the edge remains with legal-specific AIs in trustworthiness and source citation, suggesting that proprietary data access is the next competitive frontier. For law firms, corporate legal departments, and AI vendors alike, the study serves as a transparent benchmark – a rare apples-to-apples comparison — for understanding where today’s models shine and where human expertise remains indispensable. Even so, the study is weakened by the failure of the three biggest AI legal research platforms to participate. This is not the fault of Vals AI, but it leaves one wondering why the big three all opted out.