• transformer — The neural network architecture behind modern AI models, built on attention — letting every word directly consider every other word in parallel.

A convolution-based hybrid can beat highly optimized Transformers on real phone hardware, including the short-prompt regime where hybrids usually lose. The architecture was found by automated search, not hand design.

• articleHow Effectively Does A Model Use Its Memory
• articleLfm 1b Math Can Small Models Be Concise Reasoners
• articleAutomated Architecture Synthesis Via Targeted Evolution
• articleExploring Diffusion Transformer Designs Via Grafting

• articleWhich tokens does a hybrid model predict better?allenai.org
• postA 2-bit 1.8B model in 600MB, tuned for Arm SME2 siliconTencent Hy
• articleReka Edge Frontier Level Edge Intelligence For Physical AiReka AI editorial sitemap

Checking sign-in…

Loading comments…