Harvey LAB (Legal Agent Benchmark)

Harvey LAB (Legal Agent Benchmark)

Source

Vals AI (reviewed publisher results)

Version

1

Measurement date

Not reported

Metric

% · Higher is better

Evaluation scope

Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model meta/muse_spark_1_2. Configuration {"provider":"Meta","temperature":1,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 3.667; no confidence interval inferred. Fallback execution unknown. Identity basis The pinned Vals registry explicitly sets provider_endpoint=muse-spark-1.2. Atlas has route meta/muse-spark-1.2 and the same release date.

Fewer than two comparable models. No ranking is assigned. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

1. Muse Spark 1.2

   meta/muse-spark-1.2-20260805

   25.417%

   View modelEvidence independent