Intel / article

From PyTorch to the NPU: LLM-Agent-Driven Model Conversion Across Heterogeneous Inference Runtimes

Source: arxiv.org
Author: Jianhao Su, Zhanwei Wu, Chia-Heng Tu, ShengTing Huang
Date: Published Sep 24, 2026

Context

Moving a trained vision model onto a different device requires more than changing a filename. The deployment tools must convert it, handle unsupported operations and verify the resulting computation. This extension of AIPC organizes that work into stages an can execute and check, supplying instructions and scripts specific to the target runtime.

The authors report completing conversions from PyTorch to executable for representative vision models across several backends. Each deployment targets one model and one runtime. The contribution is an engineering workflow for making those conversions repeatable, including adaptation of data layout; the reported study establishes feasibility rather than a large-scale success rate across arbitrary models and devices.

Key takeaways · AI-distilled

Deployment is split into standardized, verifiable stages, with runtime-specific knowledge injected through agent skills, auxiliary scripts and staged verification loops.
A layout-adaptation and inference-replacement layer removes manual transpose insertion from the agent's repair burden.
Targets tested with representative vision models: OpenVINO for x86/NPU, RKNN for RK3588, TensorRT for NVIDIA GPUs and ONNX Runtime for a Qualcomm NPU, focused on FP16 deployment feasibility.
The authors frame the work as engineering experience and toolchain mapping, explicitly not large-scale benchmarking or a comparison of cross-runtime operator repair strategies.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Read the source arxiv.org