Vibeleaderboard
← Benchmark guide

Model results

BigCodeBench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Official source
← All evaluation groups

BigCodeBench

Source
BigCodeBench (reviewed publisher results)
Version
hard
Measurement date
Not reported
Metric
% · Higher is better
Evaluation scope
hard; complete; pass@1; prompted=True; prefill=False; result date not supplied; source captured 2026-09-10

Exact score ties share a position. Positions do not establish statistical significance. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

  1. #1DeepSeek V3

    deepseek/deepseek-chat-v3

    40.5%

    View modelEvidence independent

  2. #1R1

    deepseek/deepseek-r1

    40.5%

    View modelEvidence independent

  3. #3DeepSeek V3 0324

    deepseek/deepseek-chat-v3-0324

    35.8%

    View modelEvidence independent

  4. #4GPT-4o (2024-11-20)

    openai/gpt-4o-2024-11-20

    34.5%

    View modelEvidence independent

  5. #5GPT-4.1

    openai/gpt-4.1-2025-04-14

    33.8%

    View modelEvidence independent

  6. #6GPT-4.1 Mini

    openai/gpt-4.1-mini-2025-04-14

    31.8%

    View modelEvidence independent

  7. #6GPT-4.1 Nano

    openai/gpt-4.1-nano-2025-04-14

    31.8%

    View modelEvidence independent

  8. #8Llama 4 Maverick

    meta/llama-4-maverick-17b-128e-instruct

    29.1%

    View modelEvidence independent

  9. #9Claude 3 Haiku

    anthropic/claude-3-haiku

    19.6%

    View modelEvidence independent

  10. #10Llama 4 Scout

    meta/llama-4-scout-17b-16e-instruct

    16.9%

    View modelEvidence independent
BigCodeBench model results | VibeLeaderboard