Musings On Building A Generative Ai Product

LinkedIn’s 2024 engineering account describes a small-looking failure that repeatedly interrupted its AI assistant: model-generated tool arguments arrived in invalid formats. About 10% violated the expected schema or were not valid YAML, the structured text format used for the calls. Asking the model to repair each response added another generation step, increasing delay and computing demand. The team instead combined prompt changes with a defensive parser that detected and patched recurring mistakes, reporting a reduction to about 0.01%. That result concerns tool-call formatting, not the correctness of the assistant’s answers, and it shows that software around a model can fix a failure that prompting alone left in place.