limitations switching from Luganem to uh point one to our point two models. And so one of those things is that we basically we started on data and this is really not a crazy kind of problem. We very intuitively started from a place on a smaller scale where we were um we were basically focusing on quality versus quantity um maybe a little too much because eventually when we started scaling our models, we had to scale our training budget and with that uh came some limitations because we started hitting repetition uh like non-optimal repetition on some of our high-quality data which saturated the model a little too early. So um we One of the ways we've particularly for this like a token uniqueness problem uh, we relied on which is a very common form of synthetic data rephrasing which you know you just may heard in like Beyond Web for example. It's become pretty trendy these days. Um, and you can see here that uh you know, this is a take this ablation this is an ablation result so take the numbers with a grain of salt but what persists pretty consistently is the diff between using the orange you would be just the seeds with repetition and then the green would be replacing some of those repeated tokens with higher like with the multi-mode reverse or at least yeah, all of them or at least reducing the repetition. Um, and so uh for