Claude Code adds `claude plugin eval` to measure plugin and skill impact
- Source
- x.com
- Date
New in Claude Code: claude plugin eval See what value your plugin is adding, or if it needs more work. You can create test cases, run your plugin or skill against those test cases, score those runs, then run each case again without the plugin to see the differences.
Context
ClaudeDevs introduced claude plugin in Claude Code, Anthropic's agentic coding tool: a command that runs a defined set of test cases against a plugin or , scores the outcomes, then reruns the identical cases with the plugin removed so a developer can compare the two runs directly. Thariq, who works on Claude Code, described the same feature as a way to catch regressions when a new model ships and an existing plugin's assumptions no longer hold.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
- agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
Checking sign-in…
Loading comments…