Meet E-Commerce Bench, a new benchmark for long-horizon autonomous business operations. 🚀
Agents start with ¥100,000 to run online stores for 365 days, handling sourcing, negotiation, pricing, promotions, inventory and cash flow, in a market driven by real e-commerce data.
What's inside: 👀
- Real economics: 6886 products, 576 suppliers (152 fraudsters), 600-minute workdays, storage fees, returns and reputation.
- Long-horizon learning: almost no model learns to buy cheaper or improve its strategies over a full year of operation.
- Seven-axis evaluation: beyond year-end assets, we score six more dimensions, revealing that no single model dominates across the board.
Learn more about E-Commerce Bench: 👇
- Blog: https://t.co/fpdO9jwwiV
- Paper: https://t.co/EYIGlMPFUn
- Project: https://t.co/ss2kuThRin
- Code:
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
A year-long simulated business gives long-horizon AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → a harder target than task benchmarks, and its headline finding is that models barely learn to buy cheaper or refine strategy across the run.
