Introducing BigLaw bench: research with Harvey
BLB: Research helps us identify how foundation models are improving on research tasks, enabling investigation of ways to improve AI-based legal research.
The second of our major BigLaw Bench (BLB) expansions this quarter is BLB: Research. This dataset focused on hard agentic legal research problems. Working with our data partner Snorkel AI, a leader in creating complex expert data for frontier AI, we identified a series of US Case Law research problems that leading models are currently unable to solve — even when provided with search tools like web search.
The purpose of BLB: Research is twofold. First, to identify how foundation models are improving on research tasks and the failure modes that continue to cause them to return unsatisfactory answers. Second, to investigate ways to improve AI-based legal research through agentic systems and access to non-public data in addition to foundation model capabilities.
By enabling us to both identify the best model and to build better infrastructure for those models, BLB: Research helps us build deeper and more accurate research capabilities for our customers.
To read the article in full, click here.
