SandboxFusion
https://github.com/bytedance/sandboxfusion- Category
- AI Agents
- Rank
- No. 627Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- bytedance
- GitHub
- 1.1k stars
- Date
About
Sandbox runtime for safely executing LLM-generated code across many languages — useful for code-agent evaluation.
What it does
SandboxFusion runs submitted code, returns execution results, and judges model answers against established programming datasets. It exposes a web service plus a Python client, supports notebook execution, and handles both assertion-based tests and standard input/output comparisons.
Why it's ranked here
Its breadth is the main argument: many programming languages, GPU workloads, concurrent requests, and several evaluation formats share one service. The caution is equally concrete. Stronger isolation is optional, while the local configuration runs code without isolation.
What's good
The runner covers compiled, interpreted, hardware-description, and GPU languages. Dataset adapters handle prompt generation, code extraction, assets, assertions, and output matching. The Python client offers synchronous and asynchronous access, retries, timeouts, and batch concurrency.
Tradeoffs
The default local setup uses no execution isolation, so the security claim depends on deployment configuration. Stronger isolation relies on Linux facilities and is recommended inside Docker. Notebook mode lists only a Python kernel. The documented service and client default ports also disagree.
How to use it well
Use it for teams evaluating generated programs across languages or reproducing dataset-based coding tests. Put the service in Docker with stronger isolation, then drive repeated jobs through the Python client. It does not provide model hosting, training orchestration, or a general notebook environment.
Technical notes+
README.md documents Docker and manual Conda plus Poetry installation, with the service bound to 127.0.0.1:8080. Makefile launches sandbox.server.server:app through Uvicorn and defines pytest targets for general, CUDA, Verilog, minor-language, and online tests. pyproject.toml packages a Python service with FastAPI, Uvicorn, Pydantic, database drivers, and Transformers. docs/docs/docs/reference/config.md defines runner.isolation as none or lite; lite uses overlayfs, cgroups, and namespaces, while the shown local configuration selects none. docs/docs/docs/how-to/python-sdk.md documents the sandbox-fusion package, synchronous and asynchronous HTTP wrappers, retries, client timeouts, and concurrent execution, but defaults its endpoint to localhost:8000 rather than the server's documented port 8080.
Observed
- License
- Apache License 2.0
- Primary language
- Python
- Installation
- Docker images or a manual Conda and Poetry environment
- Interfaces
- HTTP service and installable Python SDK
- Language support
- Includes Python, C++, C#, Go, Java, NodeJS, TypeScript, Scala, Kotlin, PHP, Rust, Bash, Lua, R, Perl, D, Ruby, Julia, Verilog, and GPU execution
- Testing structure
- Pytest configuration and dedicated targets cover general, CUDA, Verilog, minor-language, and online tests
Read from README.md, Makefile, pyproject.toml, docs/README.md, docs/blog/2024-10-18-hello.md, docs/docs/api/index.md, docs/src/pages/markdown-page.md, docs/docs/docs/reference/config.md, docs/docs/docs/how-to/python-sdk.md, docs/docs/docs/how-to/add-samples.md, docs/docs/docs/background/dataset-types.md, docs/i18n/zh-Hans/docusaurus-plugin-content-blog/2024-10-18-hello.md, docs/docs/docs/how-to/use-dataset/index.md, docs/i18n/zh-Hans/docusaurus-plugin-content-docs/current/api/index.md, docs/sidebars.ts.
What it can do
Execute LLM-generated Python code safely
Python code string → Execution results and output
Execute LLM-generated JavaScript code safely
JavaScript code string → Execution results and output
Execute code in multiple programming languages
Code string and language specification → Execution results and output
Isolate code execution in secure sandbox environment
Untrusted code → Safe execution results without system access
Evaluate code agent performance
Code agent outputs and test cases → Performance metrics and evaluation results
Test AI-generated code safety
AI-generated code snippets → Safety assessment and execution logs
Tags
Tech Stack
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.