Whether the same input reliably produces the same output — something LLM systems mostly lack, which changes how you test and debug them.
Sampling, batching, and hardware differences mean an identical prompt can produce different text on consecutive calls even at temperature zero. Debugging cannot assume reproduction, and a test asserting an exact string will fail eventually for reasons unrelated to the change under test.
The workable response is to push determinism outward: assert on structure and properties rather than exact wording, make the code around the model deterministic and well tested, and treat a failure you cannot reproduce as a distribution to sample rather than a one-off.