Vibeleaderboard
← All Intel
Intel / video

Transformers.js v4.3: Structured Output in the browser

Source
youtube.com
Author
Hugging Face
Date

Context

A writes one token (a word piece) at a time, scoring every possible next piece. In a Hugging Face video, the presenter explains that , technically constrained decoding, removes the candidates that would break a rule before one is picked, so the result must match a JSON or a regular expression. Transformers.js 4.3, a library for running models in JavaScript, adds this through a new separate package that plugs into an existing at the decoding step.

In the demo, an unconstrained prompt returned Markdown wrapping some JSON. With a schema it returned only JSON and used the required field name full name. The presenter says a first attempt wrapped a Rust library in WebAssembly but was slow and needed a download of about 1 MB, so the team wrote a JavaScript-only version with help from coding agents. They describe the slowdown as minimal across tests on Gemma 4, Granite and LFM 2.5, without stating figures.

Key takeaways · AI-distilled
  • Transformers.js 4.3 adds structured output via constrained decoding: at each step, tokens that would violate a JSON schema or regex are masked out before sampling, so the model must produce output matching the exact structure.
  • The feature ships as a separate transformers structured-output package that plugs into the existing logits processor hook, so the core Transformers.js library did not need changes.
  • Hugging Face first wrapped the Rust LLM guidance library, already used by vLLM and llama.cpp, in WASM; it worked but was slow and added a roughly 1 MB download, so they rewrote it in pure JavaScript with help from coding agents.
  • In the presenter's tests across Gemma 4, Granite and LFM 2.5 tokenizers, constrained decoding added only minimal tokens-per-second overhead.
Terms in this piece · Glossary
  • Large language model — A model trained on large amounts of data to process and generate language.
  • structured output — Forcing a model's response to match a schema, so downstream code can parse it instead of guessing at prose.
  • Schema — A written description of a data format: which fields exist, what kind of value each holds and which are required.
  • agent hook — A script the harness runs automatically at a fixed point in an agent's loop — before a tool call, after an edit, on session start.
Read the source www.youtube.com
More from Hugging Face
Recommended reads
Comments

Checking sign-in…

Loading comments…