Shopify made its merchant assistant easier to maintain by delivering specialized instructions when they became relevant. Its Sidekick engineering account describes how an expanding tool collection had filled the assistant's standing instructions with special cases and conflicting guidance.
The revised approach returns task-specific guidance alongside a tool's results. The core instructions can stay focused on general behavior, while the next step receives the rules appropriate to that task. Shopify says this also lets teams change local guidance without disrupting the cached core prompt. The design addresses instruction management as an assistant gains more capabilities.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
