MiniMind
github.com/jingyaogong/minimind- Category
- AI Tools
- Rank
- No. 93Tools index
- Pricing
- Open Source
- Type
- TOOL
- Use case
- Models: Train & Run · Research & Education
- Builder
- jingyaogong
- GitHub
- 62.5k stars
- Latest release
- v2
- Date
About
A lightweight 64M parameter GPT model that can be trained from scratch in just 2 hours on a single RTX 3090. Provides complete training pipeline including pretraining, SFT, LoRA, RLHF, and tool use capabilities.
What it does
MiniMind is a hands-on language-model learning project built around readable PyTorch implementations. It lets developers inspect and run tokenization, data preparation, instruction tuning, preference optimization, reinforcement learning, tool calling, evaluation, and local chat interfaces without hiding the core training logic behind high-level frameworks.
Why it's ranked here
The strongest reason to pick MiniMind is its unusually broad, inspectable learning path. Native implementations cover dense and mixture-of-experts training, LoRA, DPO, PPO, GRPO, CISPO, and agent training. Compatibility with common model formats and inference engines makes those lessons transferable. Its small-model focus, however, favors experimentation over maximum production capability.
What's good
The repository connects algorithms to runnable workflows instead of presenting isolated examples. Training supports checkpoints, mixed precision, gradient accumulation, distributed execution, experiment logging, and multiple rollout backends. The data layer handles pretraining, supervised conversations, preference pairs, reasoning prompts, and tool metadata. Users can test models through a terminal or a Streamlit chat interface.
Tradeoffs
Setup is repository-based and dependency-heavy, with many packages pinned in one requirements list rather than a published package interface. Training speed and compatibility vary sharply across CUDA, CPU, and MPS. The included WebUI warns that thinking can become unstable during multi-turn tool use. Some demonstration tools return fixed or simulated results, so they are teaching fixtures rather than production integrations.
How to use it well
Use MiniMind to study language-model internals, reproduce a compact training sequence, or prototype custom data and alignment methods on personal hardware. Start with the supplied small datasets, verify inference, then add supervised or preference training incrementally. Use established serving engines for deployment experiments. Choose a larger model and production tool integrations when response quality or live external data matters more than inspectability.
Technical notes+
README.md documents repository cloning, requirements installation, CLI inference, Streamlit usage, distributed training, checkpoint recovery, and third-party engine compatibility. requirements.txt pins the Python dependency surface. eval_llm.py loads either native PyTorch weights or Transformers models and exposes sampling, history, MoE, LoRA, and RoPE controls. model/model_lora.py injects low-rank adapters into square linear layers and supports saving, loading, and merging adapter weights. dataset/lm_dataset.py defines pretraining, SFT, DPO, RLAIF, and agent datasets. trainer/train_dpo.py, trainer/train_ppo.py, trainer/train_grpo.py, trainer/train_lora.py, and trainer/train_agent.py implement distinct optimization paths. scripts/chat_api.py demonstrates streaming through an OpenAI-compatible endpoint, while scripts/web_demo.py provides local chat, reasoning display, and tool-call handling.
Observed
- License
- Apache License 2.0
- Primary language
- Python
- Installation
- Clone the repository and install pinned dependencies with pip from requirements.txt
- Interfaces
- Command-line inference, Streamlit WebUI, and an OpenAI-compatible HTTP service
- Model interoperability
- Supports Transformers format and documents llama.cpp, vLLM, Ollama, and Llama-Factory compatibility
- Training platforms
- Single-GPU and multi-GPU training are supported; CPU and MPS can be selected with major speed and compatibility differences
Read from README.md, requirements.txt, eval_llm.py, model/model_lora.py, scripts/chat_api.py, scripts/web_demo.py, trainer/train_dpo.py, trainer/train_ppo.py, dataset/lm_dataset.py, trainer/train_grpo.py, trainer/train_lora.py, trainer/train_agent.py.
What it can do
Train a GPT language model from scratch
Training dataset and configuration parameters → 64M parameter trained language model
Perform supervised fine-tuning on pre-trained model
Pre-trained model and supervised training dataset → Fine-tuned language model
Apply LoRA (Low-Rank Adaptation) fine-tuning
Base model and LoRA training data → LoRA-adapted model with efficient parameter updates
Execute RLHF (Reinforcement Learning from Human Feedback) training
Model and human feedback data → RLHF-optimized language model
Enable tool use capabilities in language model
Model and tool integration configuration → Language model with tool calling abilities
Run complete training pipeline on single GPU
RTX 3090 GPU and training configuration → Fully trained model in approximately 2 hours
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.