- Category
- AI Tools
- Rank
- No. 1427Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- agentscope-ai
- GitHub
- 69 stars
- Latest release
- v0.2.0
- Date
About
Multi-tenant fine-tuning framework for local LLMs with a Tinker-compatible API. Supports SFT and RL training.
What it does
TuFT runs a shared training service that accepts authenticated client requests, creates isolated sessions, performs LoRA updates, saves checkpoints, and serves generated samples. Users can connect to an existing server or start one inside a Python process from a model or YAML configuration.
Why it's ranked here
TuFT is compelling when several users need one programmable training service instead of separate scripts. Its strongest case is operational breadth: authenticated sessions, asynchronous jobs, checkpoint handling, optional Redis persistence, telemetry, embedded startup, container deployment, and cloud deployment helpers all share one client protocol.
What's good
The same client flow covers model discovery, training, optimizer updates, checkpoint saving, and sampling. API keys separate users, while configuration controls available models and storage. OpenTelemetry support covers traces, metrics, and logs. Optional Redis persistence records sessions and training metadata, with configuration checks guarding incompatible restored state.
Tradeoffs
The dependency stack is heavy and tightly constrained, including an exact PyTorch release, an exact vLLM release, and capped FastAPI and Tinker ranges. vLLM is installed only on Linux. Full backend setup adds GPU dependencies and FlashAttention, while the introductory example uses fake token IDs and leaves real tokenization to the user.
How to use it well
Choose TuFT for a team exposing shared GPU training through Tinker clients, especially when LoRA experiments must continue into checkpointed sampling. Start with the container or embedded mode, then add Redis and telemetry when operating a persistent service. It does not replace dataset preparation or tokenization, which remain client responsibilities.
Technical notes+
pyproject.toml packages TuFT with Hatchling, exposes tuft.cli:main, requires Python 3.11 through 3.14, pins Torch 2.11.0 and vLLM 0.24.0, and defines backend, persistence, examples, docs, and development extras. src/tuft/cli.py launches Uvicorn, loads YAML configuration, validates Redis configuration signatures, initializes telemetry, and exposes namespace-scoped persistence clearing. src/tuft/server.py builds the FastAPI application, applies API-key authentication, mounts an OpenAI-compatible router, queues model operations as futures, and manages service lifecycle resources. src/tuft/runtime/__init__.py discovers an existing service or starts an embedded server, then returns Tinker training and sampling clients. src/tuft/loss_fn/__init__.py registers cross-entropy, CISPO, DRO, importance-sampling, and PPO losses. src/tuft/console/console_server/main.py defines a separate FastAPI console server with configurable CORS.
Observed
- Primary language
- Python
- Packaging
- PyPI package built with Hatchling; source installation and a pre-built container image are documented
- Interfaces
- Typer CLI, Python library runtime, FastAPI HTTP service, Tinker-compatible API, and OpenAI-compatible router
- Python support
- Requires Python 3.11 or newer and earlier than Python 3.15
- Platform constraint
- The vLLM dependency is installed only on Linux; the one-command installer supports Unix platforms
- Persistence and observability
- Optional Redis persistence and OpenTelemetry traces, metrics, and logs
Read from README.md, pyproject.toml, src/tuft/cli.py, src/tuft/server.py, src/tuft/__init__.py, src/tuft/__main__.py, src/tuft/oai/__init__.py, src/tuft/loss_fn/__init__.py, src/tuft/runtime/__init__.py, src/tuft/backends/__init__.py, src/tuft/telemetry/__init__.py, src/tuft/persistence/__init__.py, src/tuft/console/console_server/main.py.
What it can do
Fine-tune local LLMs using supervised fine-tuning
Local LLM model and training dataset → Fine-tuned LLM model
Fine-tune local LLMs using reinforcement learning training
Local LLM model and RL training data → RL-trained LLM model
Manage multiple tenant fine-tuning jobs simultaneously
Multiple fine-tuning requests from different tenants → Isolated fine-tuned models for each tenant
Provide Tinker-compatible API access for fine-tuning operations
API requests in Tinker-compatible format → API responses with fine-tuning results
Process fine-tuning jobs on local infrastructure
LLM models and training configurations → Locally fine-tuned models without external dependencies
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.