← All IntelClip / OtherRCAI's bet: pre-training a 400B model in 6 months
From Local Models: Trust, Control, Optimization — Carter Abdallah, NVIDIA · ≈3:46
“we kind of uh reoriented the entire company towards let's figure out how to pre-train a you know a 400 billion parameter model in 6 months”
“now we are an open model lab working with our wonderful partners and our customers to build western open models that are permissive and you can own those and customize them or run them wherever you want”
What’s in it
- Reveals how a company pivoted to pre-train a 400B-parameter model in 6 months
- Explains the shift to becoming an open model lab for Western AI
- Highlights permissive models customers can own, customize, and run anywhere
Clip transcript
Intellect where we could probably try to pre-train ourselves. Uh so, last year we did that. We kind of uh reoriented the entire company towards let's figure out how to pre-train a you know a 400 billion parameter model in 6 months. And a lot of people said it was impossible and by in many ways it was, but we figured it out and now we are an open model lab working with our wonderful partners and our customers to build western open models that are permissive and you can own those and customize them or run them wherever you want and and that's kind of where we're at right now. So thanks for having me.
Comments
Checking sign-in…
Loading comments…