What Drives Dialectal Jailbreaks? An Ablation of Surface Form, Cultural Framing, and Strategy Banks
Source
Qingyang Xu
Author
Qingyang Xu
Date
Key takeaways · AI-distilled
A 36-cell ablation extending a classical-Chinese red-teaming framework to Shanghainese and Cantonese found dialectal surface form is neither necessary nor sufficient for high jailbreakA prompt crafted to make a model ignore its own guidelines — usually through roleplay, hypotheticals, or encoding rather than a direct request.Full definition → success.
Non-optimized English, Mandarin and naive dialect translations stayed below 8% attack success, while every condition keeping an optimizer-controlled strategy bank reached 98 to 100%.
A culture-neutral generic strategy bank hit the same ceiling at near-single-query cost, pointing to strategy-bank expressiveness, not dialect or cultural content, as the main driver. Dialect still affected query efficiency and response severity.
The author cautions that the ceiling-level success rates are sensitive to judge calibrationHow well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.Full definition → and that the design does not fully separate one-shot strategy construction from iterative search.
Terms in this piece · Glossary
jailbreak — A prompt crafted to make a model ignore its own guidelines — usually through roleplay, hypotheticals, or encoding rather than a direct request.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
Why it matters
Jailbreak success comes from the expressiveness of the prompt-strategy bank, not from obscure dialects. A culture-neutral strategy bank reaches the same ceiling at near-single-query cost, so defenses should target optimized strategies.
articleBeyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on QualityQipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu
articleDo Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code GenerationAlex Deaconu, Anubhav Gupta, Manaal Basha, Nicholas Haydu, Gema Rodr\'iguez-P\'erez
articleLatent Undertow: How Ordinary Typos Break ProbesElad David, Max Fomin, Amit LeVi