← All IntelClip / OtherGLM vs Opus real bug-fix benchmark
From Open Source Is Dead. Long Live Open Source. — Saoud Rizwan, Cline · ≈9:28
“GLM used twice as many tokens but only cost half as much.”
“Opus finished faster.”
“It used half as many tool calls.”
“GLM cleaned up dead code and verified that the build compiled before completing while Opus didn't.”
What’s in it
- Head-to-head test: GLM vs Claude Opus on a real client bug
- GLM costs half as much despite burning twice the tokens
- Opus was faster but broke the production build
Clip transcript
on the model or your end developer's responsible use of the model itself. So we recently shared uh an anecdotal experience where we were skeptical of the benchmark saying that GLM was better than Opus. So we tested them on a real bug from the client repo and while both models fixed the issue, GLM was the winner in terms of cost and code quality. So GLM used twice as many tokens but only cost half as much. Opus finished faster. It used half as many tool calls. But GLM cleaned up dead code and verified that the build compiled before completing while Opus didn't. It left a bunch of type errors and it broke the production build. Um, and so that gave us the sense that GLM was trained to spend more tokens verifying its output. Um, which is fine because the tokens are cheaper any anyways and it's really the end result that matters.
Comments
Sign in to comment.
Loading comments…