Intel / article

What Drives Dialectal Jailbreaks? An Ablation of Surface Form, Cultural Framing, and Strategy Banks

Source: arxiv.org
Author: Qingyang Xu
Date: Published Sep 29, 2026

Context

Changing a harmful request's dialect was much less consequential than changing how the attack was constructed in this controlled study. Across tests of OpenAI's GPT-4o and DeepSeek's DeepSeek-Reasoner, ordinary English, Mandarin and simple dialect translations stayed below 8% attack success. Conditions retaining an optimizer-controlled collection of prompting strategies reached 98 to 100% under the main scoring method, including a culture-neutral collection. That points to the strategy construction as the main driver of this experiment's result. The absolute rates depend on the automated judge (the paper reports a lower ceiling under a second judge), and the design did not fully separate constructing a strong initial strategy from repeatedly searching for one.

Key takeaways · AI-distilled

A 36-cell ablation extending a classical-Chinese red-teaming framework to Shanghainese and Cantonese found dialectal surface form is neither necessary nor sufficient for high success.
Non-optimized English, Mandarin and naive dialect translations stayed below 8% attack success, while every condition keeping an optimizer-controlled strategy bank reached 98 to 100%.
A culture-neutral generic strategy bank hit the same ceiling at near-single-query cost, pointing to strategy-bank expressiveness, not dialect or cultural content, as the main driver. Dialect still affected query efficiency and response severity.
The author cautions that the ceiling-level success rates are sensitive to judge and that the design does not fully separate one-shot strategy construction from iterative search.

Terms in this piece · Glossary

jailbreak — A prompt crafted to make a model ignore its own guidelines — usually through roleplay, hypotheticals, or encoding rather than a direct request.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.

Read the source arxiv.org