Qwen's E-Commerce Bench runs agents through a year of store operations
- Source
- x.com
- Date
Meet E-Commerce Bench, a new benchmark for long-horizon autonomous business operations. 🚀 Agents start with ¥100,000 to run online stores for 365 days, handling sourcing, negotiation, pricing, promotions, inventory and cash flow, in a market driven by real e-commerce data. What's inside: 👀 - Real economics: 6886 products, 576 suppliers (152 fraudsters), 600-minute workdays, storage fees, returns and reputation. - Long-horizon learning: almost no model learns to buy cheaper or improve its strategies over a full year of operation. - Seven-axis evaluation: beyond year-end assets, we score six more dimensions, revealing that no single model dominates across the board. Learn more about E-Commerce Bench: 👇 - Blog: https://t.co/fpdO9jwwiV - Paper: https://t.co/EYIGlMPFUn - Project: https://t.co/ss2kuThRin - Code:
Context
Qwen's E-Commerce Bench tests whether an learns from running a business over time. Agents receive a virtual ¥100,000 and operate stores for a simulated 365-day year, with inventory, supplier negotiations and cash-flow decisions carrying consequences across the run. The simulator is in real Taobao and Tmall data, but customer demand and supplier pricing decisions follow fixed, deterministic rules. Benchmark design.
Repeated activity rarely produced better purchasing behavior. The researchers report that across thousands of repeat purchases of the same item from the same honest supplier, 16 of the models tested showed no clear sign of pushing the purchase price down as the year went on. The therefore evaluates learning and other operating skills alongside year-end assets: a profitable result can coexist with weak adaptation. These are findings about behavior in the simulation. Learning analysis.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Checking sign-in…
Loading comments…