GradLoc traces an RL gradient spike back to a single token

Latest from Tencent HY Research blog: Bridging LLM Infra and algorithm development. 🚀   We are open-sourcing GradLoc: A white-box diagnostic tool that traces gradient spikes to the exact culprit token in O(log N) time.   Scaling RLVR no longer has to be a battle against "black-box" gradient spikes. Moving from heuristic trial-and-error to deterministic engineering diagnostics.   Blog: https://t.co/RbxXOmIsyD GitHub:

🔍 From Global Spikes to Single Tokens   Standard monitoring only tells you when a gradient spike happens. GradLoc tells you where. Using distributed binary search, it isolates the specific token causing the spike with negligible amortized overhead. It’s an "always-on" debugger for LLM training.

💡 New Collapse Mode Discovered   It’s not always "dirty data" or "training-inference mismatch." GradLoc revealed Layerwise Gradient Heterogeneity: Tokens can have "safe" Importance Sampling (IS) ratios but still cause specific layers to explode while others remain stable. Standard global clipping is too blunt a tool for this.

🛡️ The Solution: LayerClip   To address Layerwise Gradient Heterogeneity, we propose Layerwise Gradient Clipping. Instead of a uniform global clamp, LayerClip applies adaptive constraints based on local layer statistics. This fine-grained control stabilizes RLVR training where standard methods fail.