large. A vocabulary of 100,000 tokens would mean Q has 10 billion entries. One of the earliest solutions came from the D3PM paper which tackled this by building character level diffusion models. Since the character vocabulary is only a few hundred symbols, the full Q matrix could be stored and used directly. In one experiment, D3PM defined each entry Q using the similarity between characters I and J in the embedding space. But modeling language at the character level proved to be a bottleneck in quality. So this approach never really took off. Another way to handle large transition matrices is to avoid expressing them at all and instead define a closed form rule for corruption. Here's a simple one also from D3PM. This says a token will remain unchanged with probability 1 minus beta t and otherwise be replaced with a random token drawn uniformly from the vocabulary. In this formulation, pure noise looks like gibberish. But many modern diffusionbased LLMs use a slightly different approach based on a special mask token. This means a token will either stay the same or get replaced by a mask. There's no direct transition from dog to cat, just from dog to mask. So in this setup, pure noise is a fully masked out sequence. Now you might remember from the previous video that beta t controls the noise schedule. When beta is zero, all tokens stay unchanged. When beta is one, the entire sequence gets corrupted. In language, it's common to use a linear schedule, gradually increasing corruption over time. The