Right Answer, Wrong Reasons: AI Forecasts Costco - Hassan El Mghari & Omar El Mghari, Together AI

Demonstrates having models emit auditable formulas instead of doing math, and shows that a good aggregate result can hide offsetting errors, so expert checks belong in code.

• Hassan El Mghari of Together AI and his father Omar, a former CFO and head of internal audit, gave GPT-OSS-120B only Costco's public filings from before its 2024 membership fee increase, so the model could not already know the result.
• The team turned 214 filings into 3,000 facts, had the model write the financial model as structured formulas instead of doing the math itself, and ran 16 cheap attempts in parallel.
• The forecast landed within 4.4% of Costco's actual results, but an audit of its assumptions found two errors that happened to cancel out. Their conclusion: put an expert in the loop and turn the expert's checks into code.

• videoYour KV Cache Can Spot Attacks on Your AI Agents - Ari Azarafrooz, Intrinsec AI
• videoWill AI Agents Pretend to Be Human to Get Past a Check? - Maty Bohacek, Stanford
• videoThe One Thing Every AI Agent Framework Agrees On - Mislav Ivanda, Daytona
• videoAI Writes Code Faster Than Uber Can Merge It - Preetam Dwivedi & Manjari Akella, Uber

• videoThe Cheaper AI Model That Costs You More - Rayan Krishnan, Vals AIAI Engineer
• videoAI Can't Make Up a Number It Can't Look Up - Vinoo Ganesh, KeplerAI Engineer
• videoProving Alpha: How a Hedge Fund Measures AI's ROI - Hari Kumar, Contour Asset ManagementAI Engineer

Checking sign-in…

Loading comments…