Vibeleaderboard
← Benchmark guide

Model results

BigCodeBench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Official source
← All evaluation groups

BigCodeBench

Source
BigCodeBench (reviewed publisher results)
Version
hard
Measurement date
Not reported
Metric
% · Higher is better
Evaluation scope
hard; instruct; pass@1; prompted=True; prefill=True; result date not supplied; source captured 2026-09-10

Exact score ties share a position. Positions do not establish statistical significance. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

  1. #1Llama 3.3 70B Instruct

    meta/llama-3.3-70b-instruct

    28.4%

    View modelEvidence independent

  2. #2Qwen2.5 Coder 32B Instruct

    qwen/qwen-2.5-coder-32b-instruct

    27.7%

    View modelEvidence independent

  3. #3Gemma 3 27B

    google/gemma-3-27b-it

    25%

    View modelEvidence independent

  4. #3GPT-4o (2024-05-13)

    openai/gpt-4o-2024-05-13

    25%

    View modelEvidence independent

  5. #3GPT-4o (2024-08-06)

    openai/gpt-4o-2024-08-06

    25%

    View modelEvidence independent

  6. #6Phi 4

    microsoft/phi-4

    24.3%

    View modelEvidence independent

  7. #7GPT-4o-mini (2024-07-18)

    openai/gpt-4o-mini-2024-07-18

    23.6%

    View modelEvidence independent

  8. #8Llama 3.1 70B Instruct

    meta/llama-3.1-70b-instruct

    23%

    View modelEvidence independent

  9. #9Mistral Small 3

    mistralai/mistral-small-24b-instruct-2501

    21.6%

    View modelEvidence independent

  10. #9Qwen2.5 72B Instruct

    qwen/qwen-2.5-72b-instruct

    21.6%

    View modelEvidence independent

  11. #11R1 Distill Llama 70B

    deepseek/deepseek-r1-distill-llama-70b

    20.3%

    View modelEvidence independent

  12. #12Mixtral 8x22B Instruct

    mistralai/mixtral-8x22b-instruct

    18.9%

    View modelEvidence independent

  13. #13Gemma 2 27B

    google/gemma-2-27b-it

    17.6%

    View modelEvidence independent

  14. #14Llama 3.1 8B Instruct

    meta/llama-3.1-8b-instruct

    13.5%

    View modelEvidence independent

  15. #15Qwen2.5 7B Instruct

    qwen/qwen-2.5-7b-instruct

    12.2%

    View modelEvidence independent

  16. #16Llama 3.2 3B Instruct

    meta/llama-3.2-3b-instruct

    5.4%

    View modelEvidence independent

  17. #17Llama 3.2 1B Instruct

    meta/llama-3.2-1b-instruct

    0.7%

    View modelEvidence independent
BigCodeBench model results | VibeLeaderboard