How we use Insights to improve the agent behind Insights
Source
datadoghq.com
Date
Why it matters
Oversized tool results get resent in every later model request, inflating token cost. Trace-level analysis surfaces these patterns and tool-call failures so you can fix them with evidence instead of guesswork.
Key takeaways · AI-distilled
Datadog found five broad search_llmobs_spans calls across two AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → runs returned about 80,000 tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → that were then carried into model requests. A later insight recommended compact previews that keep identifiers, so the agent fetches full spans only when needed.
Task-specific details placed before tool definitions stopped providers from reusing the cached prompt prefix. Datadog recommends putting stable instructions and tool definitions first and projects about $2,734 a month in savings, which it says it has not measured.
One run made five identical invalid update_insight calls over about 18 minutes and ended in error after four retries: each call omitted the full insight object the tool required, so retrying unchanged could never succeed.
The agent guessed or altered resource names in 193 sampled sessions despite instructions to use exact names. Datadog's recommended fix moves the rule into the tool: check names against the agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → loader's list and return the valid choices on a miss.
Datadog validates each fix by replaying the affected investigations and comparing tool output size, token use and answer quality, since fewer tokens only help if the agent still investigates correctly and completely.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
