LightReasoner
https://github.com/hkuds/lightreasoner- Category
- AI Tools
- Rank
- No. 1033Tools index
Previous survey · No. 1027 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- hkuds
- GitHub
- 610 stars
- Date
About
ACL 2026 paper exploring whether small language models can teach large language models to reason, with code and training recipes.
What it does
LightReasoner compares a stronger model with a weaker but coherent one at each reasoning step. It keeps steps where their next-token distributions diverge, converts the contrast into soft supervision, then tunes the stronger model with LoRA and KL-divergence loss.
Why it's ranked here
The method targets a real fine-tuning inefficiency with a clear, reproducible mechanism. The repository reports gains across seven math benchmarks and five models, plus substantially fewer sampled problems, tuned tokens, and training time than its supervised fine-tuning comparison. Those results remain project-reported rather than independently established here.
What's good
The workflow covers data preparation, contrastive sample generation, LoRA training, adapter merging, and supporting analysis. Sampling can resume from checkpoint records. Pre-collected samples let researchers skip the expensive generation stage. The distributional labels preserve more information than a single chosen token.
Tradeoffs
Success depends on carefully balancing the model pair. The weaker model must reason coherently while remaining meaningfully behind the stronger model. Core sampling truncates distributions to the smaller vocabulary but does not verify token-ID equivalence first. Users must edit configuration placeholders, choose hardware-specific precision, and tune thresholds themselves.
How to use it well
It suits researchers testing efficient reasoning adaptation on local Hugging Face causal models, especially mathematical benchmarks. Start with the documented Qwen pairing or pre-collected samples, verify vocabulary alignment, then tune thresholds and LoRA settings cautiously. Treat it as an experiment pipeline, not a hosted inference service or end-user application.
Technical notes+
The pipeline is script-based. data_prep.py downloads GSM8K and MATH through Datasets and writes JSONL. LightR_sampling.py generates expert continuations, evaluates the weaker model at each prefix, filters steps by full-vocabulary KL divergence, builds contrastive token weights, and appends checkpoint records. It truncates both distributions to the minimum vocabulary size without first confirming ID alignment; analysis/testspace/check_vocab_alignment.py provides a separate diagnostic for that concern. LightR_finetuning.py builds dense vocabulary-sized soft labels, applies LoRA to q_proj and v_proj, and minimizes KL divergence through a custom Trainer. Precision is configurable near the top, but TrainingArguments later fixes BF16 on and FP16 off. merge.py merges the adapter into the base model. analysis/case_study.py, analysis/PPL_analysis.py, analysis/pre_analysis_stats.py, and analysis/preliminary_analysis.py provide comparison and diagnostic scripts.
Observed
- License
- MIT License
- Primary language
- Python
- Python support
- Python 3.10+
- Install surface
- Clone the repository and install dependencies from requirements.txt with pip
- Interface
- Command-line Python scripts with configuration placeholders edited in source
- Model stack
- PyTorch, Transformers, PEFT, Accelerate, Datasets, and Hugging Face Hub
- Training method
- LoRA fine-tuning with contrastive soft labels and KL-divergence loss
Read from README.md, requirements.txt, merge.py, data_prep.py, LightR_sampling.py, LightR_finetuning.py, analysis/case_study.py, analysis/PPL_analysis.py, analysis/pre_analysis_stats.py, analysis/preliminary_analysis.py, analysis/testspace/llama_KL.py, analysis/testspace/early_test.py, analysis/testspace/check_vocab_alignment.py, evaluation/manual_eval/evaluation_dataset.py, LICENSE.
What it can do
Train small language models on reasoning tasks
Training datasets and model configurations → Trained small language model specialized in reasoning
Generate reasoning demonstrations from small models
Small trained language model and reasoning problems → Step-by-step reasoning examples and solutions
Transfer reasoning knowledge to large language models
Small model outputs and large language model → Large language model with improved reasoning capabilities
Execute training recipes for model reasoning enhancement
Base models and training parameters → Enhanced models with reasoning abilities
Evaluate reasoning performance across model sizes
Trained models and reasoning benchmarks → Performance metrics and comparison results
Reproduce experimental results from research paper
Dataset and model configurations from paper → Replicated experimental outcomes and metrics
Tags
Tech Stack
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.