change? Uh now we get we will get into this mechanism. So it's kind of small atom like step meets finite precision. We need we need a two uh prerequisite. The first one two two ingredients. The ingredient one is the precision. The optimizer may keep very high precision master weights but the next four will pass really a BF6 visible view. That view has final resolution around a value of magnitude theta. uh BF16 spacing is roughly theta over 128. Uh that spacing is people call it oop basically the unit in the last piece. Basically it's the distance between the adjacent representable BF systeming value. But the update only needs to cross the near surrounding boundary to be viable to be visible. That boundary is about half of the oop. So roughly like the over 256 for weight around one. The BF systeming loop is around uh 0.78 and the near surrounding boundary is about 00039. If the optimizer notched the master weight by something like smaller than that uh the BFC visible value will run back so you will not see any change from the weight perspective rollout weight perspective. So that's the floor. The second primitive there is we call push. So for Adam maybe add here we just like we ignore the weight decay turn the per per sorry the the per parameter updates is the learning rate times a normalized direction the raw gradient can be dense and can have very different magnitudes across parameters and add like device by running uh gradients statistics. So the per wise push is usually on the order of learning rates. The paper passed I sight there they prove a bound. The addon step is at most B times the learning rates. So you do not need to actually remember the exact like bound there. Like the important notes is the adden makes the