Link: Taste answer: when code and logs arrive together, treat Find and Reconstru

Finding security flaws and reconstructing an incident require different evidence of success. AlphaSignal reports a small synthetic experiment in which some models found all the planted issues, yet none recovered the reconstruction key through bare chat. Describing an attack correctly did not establish that the model could recover the encoded material.

The earlier incident timeline supplies a separate, real example. Hugging Face's security team says the Claude models it tried first refused much of the analysis, so it ran NVIDIA's quantized build of Z.ai's GLM-5.2 on its own infrastructure. Replicating the attacker's decoding scheme recovered roughly four times as many findings as its initial scan of the raw capture. That was a result from an investigative workflow, not an isolated model benchmark.