Magika
github.com/google/magika- Category
- Developer Tools
- Rank
- No. 160Tools index
Previous survey · No. 169 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- GitHub
- 18.5k stars
- Latest release
- cli/v1.1.0
- Date
About
AI-powered file type detection tool that identifies file formats with 99% accuracy in milliseconds using a highly optimized deep learning model. Processes hundreds of billions of files weekly for Google services and supports 200+ content types.
What it does
Magika classifies a file by sampling limited portions of its bytes and running a compact model locally. Per-type confidence thresholds can replace uncertain predictions with generic text or unknown-binary labels, instead of presenting every model guess as reliable.
Why it's ranked here
Magika combines practical interfaces with unusually clear confidence handling. It runs on one CPU, keeps inference nearly constant across file sizes, supports batch and recursive use, and returns structured labels, MIME types, extensions, groups, and prediction scores.
What's good
The model loads once, then reported inference takes about five milliseconds per file. The command line accepts thousands of files, directories, or standard input. Python supports bytes, paths, and streams. Browser-side classification runs locally, while confidence modes let users tune error tolerance.
Tradeoffs
The JavaScript package is explicitly experimental, and its shown implementation supports only the strictest confidence mode. Go support remains work in progress and depends on cgo plus the ONNX Runtime library. Stream processing avoids retaining whole large files, but still traverses the complete stream.
How to use it well
Use Magika early in ingestion, upload, or security pipelines to route unknown content toward the correct parser, policy check, or scanner. It suits shell automation and embedded Python, Rust, browser, or Node workflows. It identifies content types, but does not replace malware or policy analysis.
Technical notes+
In js/magika.ts, Magika._extractFeaturesFromBytes samples trimmed beginning and ending blocks, pads ModelFeatures, runs prediction, applies overwrite_map, and falls back through per-label thresholds. The default model and config load from hosted URLs. js/magika-node.ts adds local model loading and stream support; large streams retain first and last blocks while traversing all bytes. js/postBuild.js emits CommonJS and ES module package metadata. go/cli/cli.go loads assets and a selected model from environment variables, while go/cli/cli_test.go requires cgo and ONNX Runtime build tags.
Observed
- License
- Apache 2.0
- Languages
- Rust command line tool, Python API, Rust and JavaScript/TypeScript bindings, with Go support marked work in progress
- Installation
- CLI installation through pipx, Homebrew, installer scripts, or Cargo; libraries through pip and npm
- Interfaces
- Command line interface plus Python, Rust, JavaScript/TypeScript, and Go library surfaces
- CLI output
- Human-readable, label, MIME type, JSON, JSONL, score, and custom-format output modes
- Platform support
- Homebrew instructions cover macOS and Linux; installer scripts are provided for shell and PowerShell environments
Read from README.md, docs/js.md, docs/concepts.md, js/magika.ts, js/postBuild.js, js/magika-cli.ts, js/magika-node.ts, website/vite.config.js, website-ng/astro.config.mjs, website-ng/svelte.config.js, website-ng/content.config.ts, go/cli/cli.go, go/cli/main.go, go/onnx/onnx.go, go/cli/cli_test.go.
What it can do
Detect file type from file content
File of unknown format → Identified file type/format
Classify content type across 200+ supported formats
File data → Content type classification
Analyze file format in milliseconds
File → File type identification result
Process files at scale for enterprise services
Batch of files → File type detection results for all files
Verify file extensions match actual content
File with extension → Validation of file type accuracy
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.