StepFun Open-Sources onPanda for Token-Level LLM Annotation and Debugging

Making better training examples for a language model usually means a person fixing the model's flawed answers, and the baseline the onPanda paper compares against is manual post-editing, where the annotator rewrites the response. onPanda, from researchers at StepFun, changes the unit of work to a single token, the word or word fragment a model produces one at a time. The annotator reads a response, finds the first wrong token, and either picks one of the model's own alternative tokens or types a correction. onPanda then deletes everything after that point and lets the model continue from the corrected text, repeating until the answer is acceptable.

The authors argue this produces better-matched training data, not just faster edits. Because most tokens in the final answer are still written by the model, the result stays close to what the model would naturally produce, which suits on-policy data: examples drawn from the model's own output, used for supervised fine-tuning and for preference training, where a model learns which of two answers is better. StepFun's announcement puts that closeness at a perplexity change of under 1% against the model's resampling baseline; perplexity measures how expected a text is to the model, so a small change means the corrected answer still reads like the model's own. Each correction also records an exact position with a rejected and an accepted alternative, which the paper presents as fine-grained supervision that comes paired by construction.

The speed figure is stated cautiously in the abstract: a small controlled study suggests onPanda cuts median annotation time by 52% compared with manual post-editing. StepFun says it uses the tool internally and has released it as open source, along with Panda-CVL, a dataset annotated with onPanda, and a benchmark for token-level correction. The tool can also connect to external tools and harnesses for annotating agent trajectories, and it shows each token's probability and top alternatives for debugging. The abstract and the first post of the announcement describe how the data is made; neither reports whether models trained on it outperform models trained on rewritten answers.