← All IntelClip / AI ToolsContaminated benchmarks still work as a relative reference, plus an unprompted self-review pass
From Ling 3.0 Flash First Test – A Surprisingly GOOD Coding Model! · ≈2:51
Two reusable ideas: a test that has leaked into training data is still valid for comparing models against each other, and the model volunteers a bug-hunting pass before handing back a 1,900-line file.
What’s in it
- Two reusable ideas: a test that has leaked into training data is still valid for comparing models against each other, and the model volunteers a bug-hunting pass before handing back a 1,900-line file.
Clip transcript
should probably see how this performs. So, all I've done is just set this up through open code from within the open router thing right there. Right here we have the browser OS test V2.5, which is the traditional one we run where it needs to make the GTA clone and things of the sort. This test is very likely made its way into a lot of training data by this point. However, it will still give us a reference point to see how this model compares with every other model we've tested in the last six or so months. So, we see it starts out right here just with a pretty dense to-do list about everything it's going to do. So, I'll be interested also in just seeing how this behaves throughout the process of actually creating the code, not just looking at the code itself as the end result. All right, so I like seeing that. It did generate the file very quickly, but again, if it's being hosted by someone else, especially for a model that will ideally be self-hosted, I don't care as much about speed cuz we want to see how it actually does on like something like this and not necessarily a cloud provider. However, I do like that it spit out like a 1900-line script and said, "Let me review and fix issues in the file I just generated." So, it's looking for issues, identifying bugs, and then trying to fix them. So, it's proactively going through this before we even see the result, which is nice. So, hopefully we don't have to copy-paste back and forth between developer tools.
Comments
Sign in to comment.
Loading comments…