Vibeleaderboard
← All Intel
Intel / video

Stop Fine-Tuning to Fix Retrieval Problems — Anant Srivastava

Source
youtube.com
Author
AI Engineer
Date

Context

When an assistant gets changing product facts wrong, the first question is where those facts should live. This architecture talk proposes keeping current, large, citable or access-controlled knowledge in memory the assistant can retrieve. Prompts handle small, stable behavior; model weights are reserved for reflexes that have stopped changing.

The distinction gives teams a diagnostic for choosing an intervention. A problem finding the right current document calls for work on retrieval and access to information. Teaching a model a settled, repeatable behavior is a different job. is therefore one tool within the architecture, rather than the automatic next step whenever answers disappoint.

Key takeaways · AI-distilled
  • Srivastava's diagnostic: put small, stable behavior in the prompt; put knowledge that is current, large, citable or access-controlled in memory; reserve weights for reflexes that have stopped changing.
  • His rule is to fine-tune reflexes, not facts. Medical coding is his example of a good fine-tuning target, and he says access control belongs in memory rather than in the model.
  • He argues every prompt edit, indexed document and training sample is an architecture decision, so teams that never decide where knowledge lives end up with an architecture that was accumulated rather than designed.
  • He closes with a circulating architecture in which knowledge moves between , memory and weights over time and the improves by doing its job.
Terms in this piece · Glossary
  • fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Read the source www.youtube.com
More from AI Engineer
Recommended reads
Comments

Checking sign-in…

Loading comments…