Intel / article

Qwen2.5-Coder: Code More, Learn More!

Source: qwenlm.github.io
Author: Qwen Team
Date: Published Sep 18, 2024

Context

Qwen's original Qwen2.5-Coder release specialized a general through further training on code, text linked to code, and synthetic examples. The announcement describes 5.5 trillion training , with mathematics and general-purpose material included to help retain capabilities beyond coding.

That training goal matters for a coding assistant expected to interpret instructions as well as produce programs. The release supplied 1.5B and 7B models under Apache 2.0; the 32B version was still forthcoming in this announcement. Its evaluations cover generation, completion, repair, and reasoning about code, giving several ways to assess coding usefulness rather than relying on model size alone.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Key quotes

“Today, we are excited to announce the release of the next generation of open-source coding models, Qwen2.5-Coder, and officially rename CodeQwen to Qwen-Coder.”

Read the source qwenlm.github.io