Intel / blog

Identify AI model overuse with User Insights

Source: blog.cloudflare.com
Date: Published Sep 30, 2026

Context

Cloudflare's User Insights helps organizations investigate whether they are paying for more model capability than a task needs. It classifies logged AI Gateway conversations by work type and relates those categories to model choice, cost, users and conversation length. This can reveal a pattern such as simple summarization repeatedly going to a costly . The feature description explains the analysis.

An overuse flag is a lead for investigation, not proof that a cheaper model will produce the same result. Cloudflare says the view does not automatically recommend a replacement, and teams can compare cost, latency and conversation turns for the work involved before changing defaults. Classification happens after requests finish, and analysis may trail incoming traffic by approximately one day, making this a tool for examining patterns rather than live monitoring. Classification and interpretation.

Key takeaways · AI-distilled

User Insights groups conversations into task categories such as coding, research, writing, summarization and data analysis, and its overkill view flags simple work like formatting sent to high-capability models. It does not automatically recommend a replacement model.
Classification runs asynchronously in a dedicated Worker after AI Gateway logs each request, so it adds no latency, but the dashboard can trail live traffic by about a day. Cloudflare positions it for spotting patterns over time, not live monitoring.
The classifier reads the whole conversation trajectory, including tool calls and results, and returns a category, a confidence score, and ratings for task complexity, intent ambiguity, stakes and dependence.
Turns analysis shows how much back-and-forth each task needs. Cloudflare's guidance is that a simple task that keeps taking several turns points to a prompt, model or workflow problem, since the first request is only part of the cost.
Custom apps must send a stable user_id and session_id for analysis. Putting AI Gateway behind Cloudflare Access, free for up to 50 users, lets Claude Code, Codex and OpenCode inherit identity automatically.

Terms in this piece · Glossary

reasoning model — A model trained to think — generating extended internal reasoning before answering — trading time and tokens for accuracy on hard problems.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Read the source blog.cloudflare.com