Can AI truly edit audio, not just generate it? 🎧 Tencent Hy, in collaboration…
- Source
- x.com
- Date
Can AI truly edit audio, not just generate it? 🎧 Tencent Hy, in collaboration with SJTU, SII, NTU, TJU, ZODA, PKU, FDU, and other collaborators, introduces MMAE. MMAE--A Massive Multitask Audio Editing Benchmark, is the first comprehensive evaluation benchmark for speech and audio "Banana🍌" Instead of simply requiring the AI to "generate" audio, it demands that the AI understand an existing audio clip and precisely modify it according to natural language instructions—altering what needs to be changed while leaving the rest untouched. Current models show an Exact Match Rate (EMR) below 5%, revealing a major gap in reliable audio editing. MMAE includes: ✅ 2,000 high-fidelity samples from real-world scenarios ✅ 17,741 fine-grained rubric evaluation items ✅ 7 modality settings across sound, music, speech and their mixtures ✅ 6 task complexity from basic modifications to multi-hop reasoning and multi-round editing ✅ 8 operation types across local and global granularities How to use: arXiv: https://t.co/TM81ahH7PZ GitHub: https://t.co/UR1dRUKqMD HuggingFace: https://t.co/1MHR1n3LJn Demo:
Context
MMAE, an audio-editing introduced by Tencent and collaborators, tests whether a model makes the requested change while preserving unrelated sound. Its examples include enhancing speech over background music while keeping the spoken content consistent. Each task is scored against many small rubric questions, such as whether the edit happened and whether other content survived. The paper explains this design.
The authors report an exact-match rate below 5% for the leading models they evaluated, falling to 0% on complex mixed-modality tasks. Scores come from an automated rubric-based evaluation, so the result concerns reliably completing an entire editing request rather than whether a model can make some useful change.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
postExplorationBench tests how models explore, with exactly verifiable grading
postHy Image3.5 preview is live. 🚀 Professional-grade image generation, +30% win…
postTencent Hunyuan's EvolveScaler trips up frontier models on evolving facts
postTencent Hunyuan open-sources AuK, a unified speech generation and editing model
Checking sign-in…
Loading comments…