Vibeleaderboard
Index / tool

Berkeley Function Calling Leaderboard

gorilla.cs.berkeley.edu
Visit gorilla.cs.berkeley.edu
Category
Developer Tools
Rank
No. 2439Tools index

Previous survey · No. 2228 ·

Pricing
Free
Type
TOOL
Use case
Model & Agent Evaluation
Date

About

UC Berkeley's leaderboard for how accurately models invoke functions and tools, now in its fourth version. V1 introduced AST-based scoring of the generated call, V2 added enterprise and community-contributed functions, V3 brought multi-turn interactions, and V4 extends to holistic agentic evaluation including web search. Overall accuracy is the unweighted mean of the sub-categories, and cost, latency and format sensitivity are reported beside it.

What it does

Its UC Berkeley page describes BFCL as a ranking of language models on whether they call functions and tools correctly, covering models with native tool calling and models prompted to write calls as text. Estimated benchmark cost and latency sit beside accuracy, and a live demo turns a prompt plus a function into a call.

Stated on the product site

What it measures
Accuracy of function and tool calls made by language models
Headline metric
Overall accuracy is the unweighted average of all sub-categories
Model modes
Native function-calling models and prompt-based models, labelled separately
Reported alongside accuracy
Estimated cost for the whole benchmark in USD, and latency in seconds
Format sensitivity
Tested for prompt-based models only
Reproduction
A pinned codebase checkpoint or an installable PyPI package
Submissions
Model additions and test-case contributions are requested through Discord
Publication
Paper at the International Conference on Machine Learning (ICML)

Not stated on the site

  • The page does not state how many test cases the benchmark contains or how they split across sub-categories.
  • The page says results are updated periodically but does not give a cadence.
  • The page links its code and data but does not state the license either is released under.
  • The extracted page text does not include the ranked model table itself, so which models are listed is not shown here.

Written from the product site at gorilla.cs.berkeley.edu.

Tags

benchmarkfunction-callingtool-useagentsleaderboardevaluation

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.