Intel / article

What I learned from looking at 900 most popular open source AI tools

Source: huyenchip.com
Author: Chip Huyen
Date: Published Mar 14, 2024

Context

Chip Huyen's 2024 survey offers a way to navigate AI software by the job it performs. Huyen separates infrastructure that serves models and manages resources, tools that develop or modify models, and tools that build applications around existing models. This helps distinguish changing a model itself from connecting one to a useful product. Read the survey.

The analysis covered 845 software repositories selected largely through keyword searches and a minimum of 500 stars. It maps a popular, discoverable part of the ecosystem at that time; it does not measure every AI project or establish which tools work best.

Key takeaways · AI-distilled

The list came from GitHub searches for gpt, and generative ai with a 500-star floor. 'gpt' alone returns 118K repos; the filter cut everything down to 896, of which 845 are real software and 51 are tutorials or awesome-lists.
2023's growth sat almost entirely in the application and application-development layers. Infrastructure barely moved, partly a real slowdown and partly because infra products usually are not open sourced in the first place.
The lower in the stack, the rarer the solo builder. Infra repos are nearly all organizations, over half of applications come from individual accounts, and individual-built applications average more stars than organization-built ones.
Vector databases are the one genuinely new infrastructure category, and it may not survive as a category. Vector search is old, and incumbents like Redis and DataStax are adding it where the data already lives instead of beside it.
optimization is where model-development effort concentrated. The techniques are the familiar ones, , pruning, , low-rank factorization, but the bar moved: 16-bit quantization was state of the art in 2020, now it is 2-bit and below.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.

Key quotes

“If AI feels so overwhelming right now, it’s because it is.”

“2023 was the year of AI engineering.”

“Agent is a weird category, as many agent toolings are just sophisticated prompt engineering with potentially constrained generation (e.g. the model can only output the predetermined action) and plugin integration (e.g. to let the agent use tools).”

“Applications started by individuals, on average, have gained more stars than applications started by organizations.”

Read the source huyenchip.com