Best of 2024: Open Models [LS LIVE! at NeurIPS 2024]
Source
youtube.com
Author
Latent Space
Date
Why it matters
Gives a researcher's map of how open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → models closed the gap with closed models in 2024 and where fully open releases fit, useful context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → for choosing open models today.
Terms in this piece · Glossary
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
videoWorld Models & General Intuition: Khosla's largest bet since LLMs & OpenAI
videoExtreme Harness Engineering: 1M LOC, 1B toks/day, 0% human code or review — Ryan Lopopolo, OpenAI
videoInside Abridge: The AI Listening to 100 Million Doctor Visits — Abridge's Janie Lee & Chai Asawa
videoExo: Harnesses should see their own code and logs — Alex Krentsel, UC Berekeley / Google Research
