Vibeleaderboard
Index / tool
Visit github.com
Category
Developer Tools
Rank
No. 1029Tools index
Pricing
Open Source
Type
TOOL
Latest release
v0.1.0
Date

About

TrnRT is an open-source LLM inference engine for AWS Trainium that serves an OpenAI-compatible API from one trn2.48xlarge instance. It runs decode as NKI kernels fused per layer half, keeps a prefix cache across device and host memory, and runs two engines per box behind a session-affine router. GLM-5.3-Flash is the first supported model, and the authors report 3.57M tokens per minute on the AgentX benchmark.

Tags

llm-inferenceaws-trainiumnkiopenai-apiglmpython

Tech Stack

Python

Media

TrnRT

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.