Vibeleaderboard
← Benchmark guide

Model results

BigCodeBench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Official source
← All evaluation groups

BigCodeBench

Source
BigCodeBench (reviewed publisher results)
Version
hard
Measurement date
Not reported
Metric
% · Higher is better
Evaluation scope
hard; complete; pass@1; prompted=True; prefill=True; result date not supplied; source captured 2026-09-10

Exact score ties share a position. Positions do not establish statistical significance. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

  1. #1GPT-4o (2024-08-06)

    openai/gpt-4o-2024-08-06

    36.5%

    View modelEvidence independent

  2. #2Qwen2.5 Coder 32B Instruct

    qwen/qwen-2.5-coder-32b-instruct

    33.8%

    View modelEvidence independent

  3. #3Phi 4

    microsoft/phi-4

    30.4%

    View modelEvidence independent

  4. #4GPT-4o (2024-05-13)

    openai/gpt-4o-2024-05-13

    29.1%

    View modelEvidence independent

  5. #4Qwen2.5 72B Instruct

    qwen/qwen-2.5-72b-instruct

    29.1%

    View modelEvidence independent

  6. #6Llama 3.3 70B Instruct

    meta/llama-3.3-70b-instruct

    28.4%

    View modelEvidence independent

  7. #7Llama 3.1 70B Instruct

    meta/llama-3.1-70b-instruct

    27.7%

    View modelEvidence independent

  8. #8Gemma 3 27B

    google/gemma-3-27b-it

    27%

    View modelEvidence independent

  9. #8GPT-4o-mini (2024-07-18)

    openai/gpt-4o-mini-2024-07-18

    27%

    View modelEvidence independent

  10. #10Gemma 2 27B

    google/gemma-2-27b-it

    22.3%

    View modelEvidence independent

  11. #11Mixtral 8x22B Instruct

    mistralai/mixtral-8x22b-instruct

    20.9%

    View modelEvidence independent

  12. #12R1 Distill Llama 70B

    deepseek/deepseek-r1-distill-llama-70b

    20.3%

    View modelEvidence independent

  13. #13Mistral Small 3

    mistralai/mistral-small-24b-instruct-2501

    19.6%

    View modelEvidence independent

  14. #14Qwen2.5 7B Instruct

    qwen/qwen-2.5-7b-instruct

    16.2%

    View modelEvidence independent

  15. #15Llama 3.1 8B Instruct

    meta/llama-3.1-8b-instruct

    13.5%

    View modelEvidence independent

  16. #16Llama 3.2 3B Instruct

    meta/llama-3.2-3b-instruct

    5.4%

    View modelEvidence independent

  17. #17Llama 3.2 1B Instruct

    meta/llama-3.2-1b-instruct

    1.4%

    View modelEvidence independent
BigCodeBench model results | VibeLeaderboard