How we optimized Dash's relevance judge with DSPy

Hand-tuned judge prompts break on every model swap. Optimizing the judge with DSPy against a defined objective makes migrating to cheaper models a measurable step instead of a regression risk.

• articleHow we used DSPy to turn AI evaluations into better responses in Dash chatIlya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon
• articleUsing LLMs to amplify human labeling and improve Dash search relevanceIlya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo Agriel,Ishan Mishra,Eric Wang,Dmitriy Meyerzon,Dmitriy Meyerzon
• articlePromptimus: Improving already good LLM prompts with zero manual engineeringwww.amazon.science

Checking sign-in…

Loading comments…