Model results

Harvey LAB (Legal Agent Benchmark)

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Official source

Evaluation groups

Open a group to see its exact protocol and scores. Different configurations remain separate.

No model rankings are available in this snapshot. Reported results remain available below.

Other reported results

These reports have no model ranking. Open a group for its evidence and the reason positions are withheld.

Claude Haiku 5.51
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model anthropic/claude-haiku-5-5. Configuration {"provider":"Anthropic","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","harness":null}. Reported stderr 0.716; no confidence interval inferred. Fallback execution unknown. Identity basis Exact Atlas canonical ID.
Claude Opus 4.71
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model anthropic/claude-opus-4-7. Configuration {"provider":"Anthropic","temperature":1,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","harness":null}. Reported stderr 1.998; no confidence interval inferred. Fallback execution unknown. Identity basis Exact Atlas canonical ID.
Claude Opus 4.81
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model anthropic/claude-opus-4-8. Configuration {"provider":"Anthropic","temperature":1,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","harness":null}. Reported stderr 2.148; no confidence interval inferred. Fallback execution unknown. Identity basis Exact Atlas canonical ID.
Claude Sonnet 4.61
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model anthropic/claude-sonnet-4-6. Configuration {"provider":"Anthropic","temperature":1,"top_p":null,"max_output_tokens":128000,"reasoning":true,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","harness":null}. Reported stderr 1.646; no confidence interval inferred. Fallback execution unknown. Identity basis Exact Atlas canonical ID.
Claude Sonnet 51
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model anthropic/claude-sonnet-5. Configuration {"provider":"Anthropic","temperature":1,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":"max","harness":null}. Reported stderr 1.646; no confidence interval inferred. Fallback execution unknown. Identity basis Exact Atlas canonical ID.
DeepSeek V4 Flash 07311
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model deepseek/deepseek-v4-flash-0731. Configuration {"provider":"DeepSeek","temperature":null,"top_p":null,"max_output_tokens":384000,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.148; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
DeepSeek V4 Pro 04231
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model deepseek/deepseek-v4-pro. Configuration {"provider":"DeepSeek","temperature":null,"top_p":null,"max_output_tokens":384000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 1.431; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
DeepSeek V4 Pro 08131
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model deepseek/deepseek-v4-pro-0813. Configuration {"provider":"DeepSeek","temperature":null,"top_p":null,"max_output_tokens":384000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.148; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
DeepSeek V4.1 Flash1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model deepseek/deepseek-v4.1-flash. Configuration {"provider":"DeepSeek","temperature":1,"top_p":1,"max_output_tokens":384000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 1.832; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
DeepSeek V4.1 Flash 10171
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-18 is not measurement time. Run-specific configured fallback and fallback execution are unknown. Null harness does not establish absent harness use. Performance equivalence of the caching change is not independently proved. Separate from Harvey initial results and criteria pass rate. Source model deepseek/deepseek-v4.1-flash-1017. Configuration {"provider":"DeepSeek","temperature":1,"top_p":1,"max_output_tokens":384000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.509; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3 Flash Preview1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3-flash-preview. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3.1 Flash Lite Preview1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3.1-flash-lite-preview. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3.1 Pro Preview1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3.1-pro-preview. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3.5 Flash1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3.5-flash. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.833; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3.5 Flash Lite1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3.5-flash-lite. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3.6 Flash1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3.6-flash. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 1.174; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3.7 Flash1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3.7-flash. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.148; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Gemini 3.8 Flash1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model google/gemini-3.8-flash. Configuration {"provider":"Google","temperature":1,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.415; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-5.41
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-5.4-2026-03-05. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Identical provider, model version, and literal snapshot date. Atlas encodes that date without hyphens. Preserve source xhigh reasoning.
GPT-5.4 Mini1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-5.4-mini-2026-03-17. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Identical provider, model version, and literal snapshot date. Atlas encodes that date without hyphens.
GPT-5.4 Nano1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-5.4-nano-2026-03-17. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Identical provider, model version, and literal snapshot date. Atlas encodes that date without hyphens.
GPT-5.51
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-5.5. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 1.174; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-5.6 Luna1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-5.6-luna. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.833; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-5.6 Sol1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-5.6-sol. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.833; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-5.6 Terra1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-5.6-terra. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.833; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-6 Astra1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-6-astra. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 1.174; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-6 Luna1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-6-luna. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 1.362; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-6 Sol1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-6-sol. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.823; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
GPT-6.1 Sol1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model openai/gpt-6.1-sol. Configuration {"provider":"OpenAI","temperature":null,"top_p":null,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 1.755; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Inkling1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model thinkingmachines/inkling. Configuration {"provider":"Thinkingmachines","temperature":1,"top_p":1,"max_output_tokens":256000,"reasoning":null,"reasoning_effort":"0.99","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.833; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Inkling Small1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model thinkingmachines/inkling-small. Configuration {"provider":"Thinkingmachines","temperature":1,"top_p":1,"max_output_tokens":262000,"reasoning":null,"reasoning_effort":"0.99","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.833; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Mercury 2.51
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model inception/mercury-2.5. Configuration {"provider":"Inception","temperature":null,"top_p":null,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"high","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
MiMo-V2.51
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model xiaomi/mimo-v2.5. Configuration {"provider":"Xiaomi","temperature":1,"top_p":0.95,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
MiMo-V2.5-Pro1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model xiaomi/mimo-v2.5-pro. Configuration {"provider":"Xiaomi","temperature":1,"top_p":0.95,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0.833; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
MiMo-V2.6-Flash1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model xiaomi/mimo-v2.6-flash. Configuration {"provider":"Xiaomi","temperature":1,"top_p":0.95,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.543; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
MiMo-V2.6-Pro1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model xiaomi/mimo-v2.6-pro. Configuration {"provider":"Xiaomi","temperature":1,"top_p":0.95,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.453; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Muse Spark 1.21
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model meta/muse_spark_1_2. Configuration {"provider":"Meta","temperature":1,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 3.667; no confidence interval inferred. Fallback execution unknown. Identity basis The pinned Vals registry explicitly sets provider_endpoint=muse-spark-1.2. Atlas has route meta/muse-spark-1.2 and the same release date.
Muse Spark 1.31
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model meta/muse_spark_1_3. Configuration {"provider":"Meta","temperature":1,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 3.416; no confidence interval inferred. Fallback execution unknown. Identity basis The pinned Vals registry explicitly sets provider_endpoint=muse-spark-1.3. Atlas has route meta/muse-spark-1.3 and the same release date. The source xhigh configuration remains explicit.
Muse Spark Open1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model meta/muse-spark-open. Configuration {"provider":"Together AI","temperature":1,"top_p":1,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 3.094; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Muse Wonder1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model meta/muse-wonder. Configuration {"provider":"Meta","temperature":1,"top_p":null,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":"max","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 3.103; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Nemotron 3 Ultra1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model nvidia/nemotron-3-ultra-550b-a55b. Configuration {"provider":"Nvidia","temperature":1,"top_p":0.95,"max_output_tokens":128000,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Exact source ID is an explicit route on this single Atlas release.
Nemotron 3.5 Lightning1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model fireworks/nemotron-lightning-3p5-30b-a3b. Configuration {"provider":"Fireworks AI","temperature":1,"top_p":0.95,"max_output_tokens":131072,"reasoning":null,"reasoning_effort":null,"verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 0; no confidence interval inferred. Fallback execution unknown. Identity basis Vals model metadata directly links the exact Hugging Face checkpoint stored on this Atlas release. The serving provider remains explicit.
Qwen3.8 27B1
Harvey LAB (Legal Agent Benchmark) · Vals AI (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. …
Vals overall task pass percentage, the mean of two judges after each requires all criteria to pass. Source benchmark version 1. Split unresolved: prose says held-out; machine dataset_type says public. GPT 5.5 judge uses medium reasoning; Claude Sonnet 4.6 judge is unmodified per Vals. Valkyrie, Vals model library, Harvey generation/grading protocol. Internet disabled. Tools: Read File, Edit File, Write File, Glob, Bash, Grep. Skills: docx, pptx, xlsx. DOCX redline grading repair reported, rubric unchanged. Judge prompt split for caching, with content otherwise unchanged per operator. Exact run commit, measurement date, run count and dataset revision unknown; result date not supplied. Page update 2026-10-16 is not measurement time. Registry absence does not establish absent fallback execution. Separate from Harvey initial results and criteria pass rate. Source model alibaba/qwen3.8-27b. Configuration {"provider":"Alibaba","temperature":1,"top_p":0.95,"max_output_tokens":65536,"reasoning":null,"reasoning_effort":"xhigh","verbosity":null,"compute_effort":null,"harness":null}. Reported stderr 2.415; no confidence interval inferred. Fallback execution unknown. Identity basis Vals model metadata directly links the exact Hugging Face checkpoint stored on this Atlas release. The serving provider remains explicit.