AI Scientist
https://github.com/sakanaai/ai-scientist- Category
- AI Agents
- Rank
- No. 958Tools index
Previous survey · No. 943 ·
- Pricing
- Open Source
- Type
- AGENT
- Builder
- sakanaai
- GitHub
- 14.5k stars
- Date
About
Sakana AI's framework for fully automated open-ended scientific discovery — LLMs that propose hypotheses, run experiments, and write papers end-to-end.
What it does
AI Scientist starts from a domain template containing baseline code, prompts, and seed ideas. It iteratively develops candidate studies, checks literature novelty, edits experiment code, runs bounded trials, creates plots, compiles a LaTeX paper, and asks a model to critique the result.
Why it's ranked here
This is a serious research automation prototype, not a generic agent wrapper. Its strongest evidence is the concrete pipeline: repeatable templates, baseline comparisons, bounded experiment retries, literature search, PDF production, and structured criticism. The verdict stays mixed because safe execution and suitable GPU infrastructure remain the operator’s responsibility.
What's good
The workflow preserves useful research artifacts at each stage, including copied experiment code, numerical summaries, notes, plots, papers, and structured criticism. Failed or timed-out trials feed their errors back into another editing attempt. Literature search supports two providers, and experiment ideas can run concurrently across several GPUs.
Tradeoffs
The supplied domains are narrow: language modeling, low-dimensional diffusion, and grokking. Community templates are explicitly unmaintained. Current workloads target Linux, NVIDIA GPUs, CUDA, and PyTorch; CPU execution is described as impractical. The system runs model-written code that may install dangerous packages, access the web, or spawn processes, so isolation is mandatory.
How to use it well
Use it as a supervised research harness when you already have a reproducible baseline, measurable outputs, GPU capacity, and a tightly scoped template. Inspect generated code, logs, results, citations, and reviews before trusting conclusions. It does not supply a safe execution boundary or replace expert validation of scientific claims.
Technical notes+
launch_scientist.py exposes the CLI, selects a template under templates, calls ai_scientist/generate_ideas.py for iterative JSON idea generation and Semantic Scholar or OpenAlex novelty checks, then delegates code edits through Aider. ai_scientist/perform_experiments.py permits up to five runs, retries editing after failures, deletes failed run directories, and invokes plotting after experiments. ai_scientist/perform_writeup.py checks citations, figures, duplicate sections, and LaTeX diagnostics before running pdflatex and bibtex. ai_scientist/perform_review.py extracts PDF text and can ensemble several structured reviews into a meta-review. ai_scientist/llm.py provides provider-specific clients and retry handling. requirements.txt installs API clients, PyTorch, data tooling, plotting, PDF parsing, Aider, and experiment tracking dependencies.
Observed
- Primary language
- Python
- Install surface
- Conda environment with Python 3.11, pip requirements, and a system installation of TeX Live
- Interface
- Command-line experiment runner plus importable Python review components
- Platform target
- Linux with NVIDIA GPUs, CUDA, and PyTorch
- Included research templates
- NanoGPT, 2D diffusion, and grokking
- External services
- Supports model APIs from OpenAI, Anthropic, DeepSeek, OpenRouter, and Google, plus Semantic Scholar or OpenAlex literature search
Read from README.md, requirements.txt, launch_scientist.py, ai_scientist/llm.py, ai_scientist/generate_ideas.py, ai_scientist/perform_review.py, ai_scientist/perform_writeup.py, review_iclr_bench/iclr_analysis.py, ai_scientist/perform_experiments.py.
What it can do
Generate scientific hypotheses
Research domain or existing scientific literature → Novel testable hypotheses
Design and execute experiments
Scientific hypothesis and experimental parameters → Experimental results and data
Analyze experimental data
Raw experimental data and results → Statistical analysis and interpretations
Write scientific papers
Experimental results, data analysis, and research findings → Complete scientific papers with methodology, results, and conclusions
Conduct literature review
Research topic or field of study → Comprehensive review of existing scientific literature
Propose new research directions
Current scientific knowledge and experimental outcomes → Suggestions for future research avenues
Intel on AI Scientist
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.