← All IntelClip / EducationThe benchmark jump Muse Spark 1.1 claims over 1.0
From Meta Muse Spark 1.1 Hands-On Review · ≈0:40
Sets the headline claim under test — a five-fold jump on Deep SWE 1.1 with a smaller gain on SWE-Bench Pro — which the rest of the hands-on session is built to check.
What’s in it
- Sets the headline claim under test — a five-fold jump on Deep SWE 1.1 with a smaller gain on SWE-Bench Pro — which the rest of the hands-on session is built to check.
Clip transcript
I do just want to start out by taking a quick look at the introduction post. Of course, this does seem like a significant leap in capability over its predecessor based off of what we see in these benchmark charts right here. Now, for the purpose of this, I'm going to for this moment ignore the other models here. But look at the base one and look at the jumping capability on Deep S. SWE 1.1. It went from a 10 to a 53.3. Sweetbench Pro went up. Not as much of a significant leap, but still this does seem like a proper jump over its predecessor. And for that, I am excited. Now, because this is compared to other
Comments
Sign in to comment.
Loading comments…