← All IntelClip / EducationKernel-free hardware makes LLM-written low-level code work
From "An endless demand for compute" | Jonathan Ross, founder of Groq · ≈19:27
“I think it may already have gotten good enough, but it's not as binary as it sounds.”
“Well, just like AI will get better over time and will produce better kernels, if the hardware that you're compiling to is easier to reason about, it will produce even better kernels.”
“So, we've actually been using AI to program the LPUs and been getting very good results because it's so easy to to wrap your head or in this case your LLM around the problem.”
“you're going to see a lot of people trying to do chips because it's going to be very easy to design one but it's very hard to take one to production.”
What’s in it
- Ties AI codegen quality directly to hardware design: an architecture that is easier to reason about yields better machine-written kernels, so simplicity compounds.
Clip transcript
>> Speaking about AI, I want to talk about its impact on engineers in particular. Kuda kernels are notoriously hard to write manually. Do you think AI is going to get good enough to write them itself? >> I think it may already have gotten good enough, but it's not as binary as it sounds. >> What is what is good enough? Well, you don't write a kernel or not write a kernel. It's how good is the kernel? How efficient is it? How performant is it? How easily does it lead or or fuse with other kernels? Um how general is it? How reusable? or how many different kernels for special use cases. And um what you're going to start seeing is that uh as AI gets better and better, the the kernels will get better and better, but the more time you spend working on a particular kernel, the better that kernel will get. >> Now, one of the interesting things about the Grock architecture, the the LPUs, is that um we actually are have a kernel free architecture. And the inspiration here was when Gro was started, we didn't have LLMs to write the software. We had to do it all ourselves. We had a small team. So we built a chip that didn't have a lot of the the complexity around compiling to it. Um you basically it's just an easier problem. Well, just like AI will get better over time and will produce better kernels, if the hardware that you're compiling to is easier to reason about, it will produce even better kernels. >> So, we've actually been using AI to program the LPUs and been getting very good results because it's so easy to to wrap your head or in this case your LLM around the problem. >> That's very interesting. So, zooming out, AI has lowered the barrier to writing software. From what you're saying, it sounds like that's starting to happen for hardware as well. Will we see more people building hardware because it's easier? >> Absolutely. I think you'll see more people trying to design hardware. One thing that I think is going to be a problem though is hardware is a physical thing. It requires experiments. And whereas in software you see the the sort of result instantaneously and you can iterate in hardware there are supply chains and there are big bets. And so one of the things that's that's you know you're going to see is you're going to see a lot of people trying to do chips because it's going to be very easy to design one but it's very hard to take one to production. And so it's going to become the baby turtle problem where and there's only a finite amount of you know supply in the world. You know when a customer is making a bet they want to bet on something they know is going to work. So whereas before LLMs made it easy to write software or write uh you know RTL for for chips which is the software for chips. Um there were a relatively small number of companies doing it. I think you'll see more, but I think you'll see an even smaller number going to production because it'll be so hard to to choose between. You'll just want to go with the ones that you can depend on. >> Sounds very similar to software actually. It's very easy to build a
Comments
Checking sign-in…
Loading comments…