Build Tool-Calling Parsers in Rust with AI Agents
- Source
- developer.nvidia.com
- Author
- Keiven Chang
- Date

Context
Model families write tool calls in their own markup, and something has to turn that stream of back into a structured call an application can run. NVIDIA calls that piece a frontend parser, and its developer post shows how to have an write one for a new model using frontend-crates, the Rust components behind the tool-calling parsers in NVIDIA Dynamo, its framework. The parser reads token IDs or decoded text and separates visible text, reasoning and tool calls in order, and an adapter turns those events into an OpenAI-compatible response. The hard part is finding where each piece starts and ends while output is still . A marker can be split across chunks, so emitting too early leaks markup into the answer and waiting too long delays ordinary text. The post separates two failures: a call cut off mid-argument, which the parser must handle by documented recovery rules rather than guessing the rest, and a well-formed call to a tool that was never offered, which the parser reports as received while the serving layer checks availability, authorization and schema.
Correct behavior is defined by a shared Conformance Corpus: language-neutral YAML files that pair a fixed input with the expected output, called GOLDEN. Its UNIFIED cases cover complete, malformed, truncated, multiple and interleaved calls, plus streaming boundaries and end of input, and the post shows one weather request written in the native formats of Beam, GPT-OSS, Muse Spark and NVIDIA Nemotron 3, all expected to yield the same get_weather call for Paris. For a new model, the agent adapts those cases to the model's grammar and then stops for a human to review the expectations against the format specification, the reference implementation and other parsers such as vLLM and SGLang. Only after approval does it write the parser, run the tests and produce a local conformance report. The authors stress the order because a green result only means the output matches GOLDEN, so wrong expectations let a parser pass and still misbehave, and any change the agent proposes to expected behavior needs the same review.
Passing those tests is described as a starting point. The post asks for checks of speed, memory and streaming, including replaying fixtures split at every position and cutting the stream off inside a marker, since a parser can return the right final answer and still make the caller wait until the end. Guided decoding, which constrains generation to the model's markers and argument schema, can reduce malformed calls, but NVIDIA notes it cannot make the model pick the right city or prevent truncation, so the parser's recovery checks stay.
- NVIDIA's frontend-crates parsers treat a call naming a tool that was not offered as a semantic error: the parser reports the call as generated and the serving layer checks availability, authorization and schema, because renaming it would hide what the model produced.
- Streaming makes incomplete calls hard to judge because a marker or argument can split across chunks: emitting too early leaks syntax into the answer, waiting too long delays ordinary text. The parsers follow documented recovery rules instead of guessing intent.
- A shared Conformance Corpus of language-neutral YAML fixtures pairs model-specific inputs with one expected GOLDEN result, so the same get_weather behavior is tested across Beam, GPT-OSS, Muse Spark and Nemotron 3 despite their different markers.
- The authors say agent-generated GOLDEN tests need case-by-case human review against the model spec, reference implementation and parsers such as vLLM and SGLang, because wrong expectations let a parser pass every test and still behave incorrectly.
- Guided decoding can reduce malformed calls by constraining generation to the model's markers and argument schema, but the authors note it cannot prevent truncation or a wrong argument choice, so parser recovery checks remain necessary.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
- streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.
Checking sign-in…
Loading comments…


