On The Challenges Of Using Black Box Apis For Toxicity Evaluation In Research 2023 04 24
Source
Cohere editorial sitemap
Author
Cohere editorial sitemap
Date
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
determinism — Whether the same input reliably produces the same output — something LLM systems mostly lack, which changes how you test and debug them.
Why it matters
Safety evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → built on a commercial classifier silently drift when the vendor updates it, so numbers reported months apart are not comparable. This is a trap for anyone tracking toxicity regressions over time.
articleBam Just Like That Simple And Efficient Parameter Upcycling For Mixture Of Experts 2024 08 15
articleMultilingual Arbitrage Optimizing Data Pools To Accelerate Multilingual Progress 2024 08 28
articleOne Tokenizer To Rule Them All Emergent Language Plasticity Via Multilingual Tokenizers 2025 05 30
articleInclude Evaluating Multilingual Language Understanding With Regional Knowledge 2024 11 29