embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
Long system prompts are a recurring inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → tax on self-hosted agents; distilling them into learned gist token embeddingA list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.Full definition → cut a 6k-token prompt 4:1 with measured latency and throughput gains and no quality loss.
