OpenRouter now runs inside @render Workflows.
Stop putting LLM batches in your web service. Each prompt becomes its own task run, queued, retried, and fanned out by Render, with the model call routed by OpenRouter.
A web service has to hold an HTTP request open for the whole batch.
A Workflow task keeps running after that request ends, retries on its own, and fans out one run per prompt, so a single slow call never blocks the rest.
Render provisions compute for those runs on demand and deprovisions it when they finish, so the fan-out needs no pre-provisioned worker pool.
Each run calls Auto Router with openrouter/auto, and every result keeps completion.model, so you can see which model OpenRouter picked for each prompt.
One caveat worth knowing: a retry can repeat a billable call if the first request succeeded but the task failed before returning, so add idempotency for production side effects.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
A concrete pattern for LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → fan-out: per-prompt task runs that outlive the HTTP request, compute provisioned on demand, per-run model attribution via completion.model, and the billing trap where a retry can repeat a paid call.
