- Category
- Developer Tools
- Rank
- No. 1029Tools index
- Pricing
- Open Source
- Type
- TOOL
- GitHub
- 1.6k stars
- Latest release
- v0.1.0
- Date
About
TrnRT is an open-source LLM inference engine for AWS Trainium that serves an OpenAI-compatible API from one trn2.48xlarge instance. It runs decode as NKI kernels fused per layer half, keeps a prefix cache across device and host memory, and runs two engines per box behind a session-affine router. GLM-5.3-Flash is the first supported model, and the authors report 3.57M tokens per minute on the AgentX benchmark.
Tags
llm-inferenceaws-trainiumnkiopenai-apiglmpython
Tech Stack
Python
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.