Intel / article

NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction

Source: arxiv.org
Author: NCP Team, Jiaqi Cao, Chiyu Chen, Shuang Cheng, Xu Cheng, Beiya Dai, Yufan Feng, Kewen Ge, Ruijun Ge, Jiayi Huang, Yang Jiao, Dahua Lin, Zhouhan Lin, Yifan Liu, Yuliang Liu, Biqing Qi, Mowen Ruan, Junzhe Shen, Yunchong Song, Hao Sun, Zhongbo Tian, Yixuan Wang, Rubin Wei, Jiaxin Xiong, Kangyu Yang, Qian Yao, Qi Zhang, Bowen Zhou
Date: Published Sep 11, 2026

Context

NCP-ArchPreview trains a to predict internal representations of longer text spans alongside its usual prediction of the next text unit. These learned representations, called concepts, are constructed from the model's internal states. A dedicated component predicts future concepts and feeds them back into text generation, while the model continues producing text one unit at a time.

The authors report that their 8.9-billion-parameter model approaches the training loss of a same-sized baseline using 85% of its computation. Training loss measures performance on the training objective; this comparison supports investigating a more efficient training method, without establishing an equivalent reduction in the cost of answering users' questions.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Read the source arxiv.org