Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains

A small, permissively licensed model aimed at the intermediate calls in agent systems — routing, tool selection, summarization, validation — where frontier pricing is waste, and self-hostable for proprietary code.

• mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
• RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
• subagent — A separate agent spawned by another to do one scoped piece of work in its own context, returning only the result.

• articleQwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated ParametersQwen Team
• articleSolar Pro 3 0323Upstage
• articleSwitch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient SparsityWilliam Fedus et al.

Checking sign-in…

Loading comments…