Intel / article

ERNIE 4.5 Gets a Major Inference Speed Boost

Source: unknown
Date: Published Sep 12, 2025

Context

Baidu's PLAS update makes long-document processing faster by selecting which blocks of text receive . A small learned selector identifies relevant blocks, reducing the calculations while leaving the original model weights unchanged. The release supports two ERNIE 4.5 Paddle models deployed through FastDeploy. The announcement reports that the smaller model's end-to-end latency fell from 61.400 to 42.157 seconds on a long-book summarization test averaging roughly 113,000 input .

The reported accuracy tradeoff was small on two separate long- benchmarks. That supports PLAS as an option for these deployments, with the measured speed gains tied to the tested workload.

Terms in this piece · Glossary

attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

Source link unavailable.unknown