Harvey LAB (Legal Agent Benchmark)

Harvey LAB (Legal Agent Benchmark)

Source

Vals AI (reviewed publisher results)

Version

1

Measurement date

Not reported

Metric

% · Higher is better

Evaluation scope

Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model deepseek/deepseek-v4-pro-0813. Configuration {"provider":"DeepSeek","temperature":null,"top_p":null,"max_output_tokens":384000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.148; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.

Fewer than two comparable models. No ranking is assigned. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

1. DeepSeek V4 Pro 0813

   deepseek/deepseek-v4-pro-20260813

   7.5%

   View modelEvidence independent