NL2Repo-Bench
github.com/multimodal-art-projection/nl2repobench- Category
- Developer Tools
- Rank
- No. 3185Tools index
Previous survey · No. 3057 ·
- Listed in
- Find AI benchmarks
- Type
- TOOL
- Use case
- Model & Agent Evaluation
- GitHub
- 188 stars
- Date
About
Tests whether coding agents can build complete Python repositories from natural-language specifications, scored against the projects’ test suites.
What it can do
Evaluate coding agents by generating a complete Python repository from a natural-language specification
Natural-language specification → Python repository
Score generated repositories against the project's test suite
Generated Python repository and test suite → Test results/score
Why it made the leaderboard
Compare the task, benchmark version, harness, and grading method before using model scores to choose a model.
Tags
benchmarkevaluation
Tech Stack
Python
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.