Vibeleaderboard
← Benchmark guide

Model results

BigCodeBench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Official source
← All evaluation groups

BigCodeBench

Source
BigCodeBench (reviewed publisher results)
Version
full
Measurement date
Not reported
Metric
% · Higher is better
Evaluation scope
full; instruct; pass@1; prompted=True; prefill=True; result date not supplied; source captured 2026-09-10

Exact score ties share a position. Positions do not establish statistical significance. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

  1. #1GPT-4o (2024-05-13)

    openai/gpt-4o-2024-05-13

    51.1%

    View modelEvidence independent

  2. #2Qwen2.5 Coder 32B Instruct

    qwen/qwen-2.5-coder-32b-instruct

    49%

    View modelEvidence independent

  3. #3Llama 3.3 70B Instruct

    meta/llama-3.3-70b-instruct

    46.9%

    View modelEvidence independent

  4. #4GPT-4o-mini (2024-07-18)

    openai/gpt-4o-mini-2024-07-18

    46.1%

    View modelEvidence independent

  5. #4Llama 3.1 70B Instruct

    meta/llama-3.1-70b-instruct

    46.1%

    View modelEvidence independent

  6. #6Qwen2.5 72B Instruct

    qwen/qwen-2.5-72b-instruct

    45.8%

    View modelEvidence independent

  7. #7Phi 4

    microsoft/phi-4

    45.5%

    View modelEvidence independent

  8. #8Mistral Small 3

    mistralai/mistral-small-24b-instruct-2501

    45.3%

    View modelEvidence independent

  9. #9Gemma 2 27B

    google/gemma-2-27b-it

    42.8%

    View modelEvidence independent

  10. #10Mixtral 8x22B Instruct

    mistralai/mixtral-8x22b-instruct

    40.6%

    View modelEvidence independent

  11. #11Qwen2.5 7B Instruct

    qwen/qwen-2.5-7b-instruct

    37.6%

    View modelEvidence independent

  12. #12R1 Distill Llama 70B

    deepseek/deepseek-r1-distill-llama-70b

    35.3%

    View modelEvidence independent

  13. #13Llama 3.1 8B Instruct

    meta/llama-3.1-8b-instruct

    32.8%

    View modelEvidence independent

  14. #14Llama 3.2 3B Instruct

    meta/llama-3.2-3b-instruct

    23.4%

    View modelEvidence independent

  15. #15Llama 3.2 1B Instruct

    meta/llama-3.2-1b-instruct

    8.2%

    View modelEvidence independent
BigCodeBench model results | VibeLeaderboard