Gemma PyTorch
https://github.com/google/gemma_pytorch- Category
- AI Tools
- Rank
- No. 2242Tools index
Previous survey · No. 2169 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- GitHub
- 5.7k stars
- Date
About
Google's official PyTorch implementation of the Gemma open-weight models — load, fine-tune, and serve from idiomatic PyTorch.
What it does
Gemma PyTorch turns downloaded model checkpoints into local text and image-aware generation. Its model code handles attention, token sampling, caching, optional integer quantization, and image preprocessing. Separate execution paths target standard PyTorch and distributed PyTorch/XLA environments.
Why it's ranked here
This is a strong reference implementation for understanding and running several Gemma generations without hiding the model mechanics. It covers text-only and multimodal inference across major accelerator classes. The narrow inference focus and external checkpoint setup keep it from being a complete model development platform.
What's good
The repository exposes meaningful implementation detail, including rotary embeddings, local sliding attention, key-value caches, top-k and top-p sampling, per-channel integer quantization, and model parallel collectives. Configuration covers multiple Gemma generations and sizes. Multimodal support accepts interleaved text and images, including multiple images in one prompt.
Tradeoffs
The core models explicitly describe themselves as inference-only, and the supplied examples focus on generation rather than training. Users must obtain checkpoints and tokenizer assets separately. Dependencies are pinned to exact versions, while installation requires Python 3.11 or newer. Docker and accelerator-specific setup add operational work. Google also disclaims official product support.
How to use it well
Use it when you want readable PyTorch model internals, local checkpoint inference, or experiments spanning CPU, CUDA, TPU, and multimodal prompts. Start with the container examples and match each checkpoint to its configured variant. Choose another tool for managed serving, a documented training pipeline, or vendor-backed production support.
Technical notes+
setup.py packages gemma, requires Python >=3.11, and reads exact dependencies from requirements.txt, including torch 2.6.0, numpy 2.2.3, Pillow 11.1.0, SentencePiece 0.2.0, and absl-py 2.1.0. gemma/model.py implements inference with KV caching, rotary embeddings, global or sliding attention, greedy or temperature sampling, top-p and top-k filtering, and optional int8 weights with per-channel scalers. gemma/config.py defines Gemma 1, 2, and 3 variants, including SigLIP vision configuration for multimodal sizes. gemma/gemma3_model.py combines text embeddings with vision embeddings, while gemma/gemma3_preprocessor.py handles interleaved images, pan-and-scan crops, placeholder tokens, batching, and padding. scripts/run.py and scripts/run_multimodal.py provide Abseil-based command-line examples for CPU or CUDA. scripts/run_xla.py drives distributed XLA or CUDA generation through gemma/xla_model_parallel.py. Notably, gemma/model_xla.py imports Gemma modules through a google3.third_party namespace rather than the local package namespace used elsewhere.
Observed
- License
- Apache License 2.0
- Primary language
- Python
- Packaging
- Setuptools package named gemma, requiring Python 3.11 or newer
- Dependencies
- Exact pins for PyTorch, NumPy, Pillow, SentencePiece, and Abseil Python
- Interfaces
- Python library plus command-line inference scripts
- Platform support
- CPU, CUDA GPU, and TPU through PyTorch and PyTorch/XLA
- Model inputs
- Text-only and multimodal generation with interleaved text and images
Read from README.md, setup.py, requirements.txt, gemma/model.py, scripts/run.py, gemma/config.py, gemma/__init__.py, gemma/model_xla.py, gemma/tokenizer.py, scripts/run_xla.py, gemma/gemma3_model.py, scripts/run_multimodal.py, gemma/xla_model_parallel.py, gemma/gemma3_preprocessor.py.
What it can do
Load pre-trained Gemma models
Model identifier or path → Loaded PyTorch model instance
Fine-tune Gemma models on custom data
Training dataset and model parameters → Fine-tuned Gemma model
Serve Gemma models for inference
Text prompts or queries → Generated text responses
Generate text from natural language prompts
Text prompt → AI-generated text completion
Export trained models
Trained Gemma model → Saved model files in PyTorch format
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.