Morgan Stanley's ALPHALAB: Multi-Agent Research Across Optimization Domains — Brendan Rappazzo
- Source
- youtube.com
- Author
- AI Engineer
- Date

Context
Morgan Stanley's ALPHALAB research system is shifting toward tests designed by human experts before agents begin optimizing models. The team's presentation explains the problem with its earlier approach: an agent could build an evaluation, then spend substantial effort improving against a flawed test.
The described successor uses controlled environments where agents submit packaged models and receive scores. Researchers can separately inspect performance on held-out data that the agent does not see. This separates the feedback used to guide experimentation from the evidence used to judge its result. The system's proposed self-improvement depends on that measurement discipline; the talk does not establish realized trading returns.
- multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
- Context and context window — Context is the information available for a generation; the context window is the capacity available to hold it.
- Harness — The software that sends a model its context, recognizes tool calls, runs or refuses them, returns results and keeps the record.
- fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
“it really felt possible for the first time with like with Opus 4.5 and with, you know, these harnesses like Claude Code and Codex”
“LLMs aren't malicious, but they can make, you know, very silly mistakes and if you're optimizing against a bad a bad eval, the whole thing kind of falls apart.”
“this is a lesson I learned over and over, like monthly, working with these models, you have to start with good eval.”
“to me evals and environments are the same thing. It's just you train in environments.”
“I think we're already seeing it with GLM 5.2, and so I really think all of your value as like an enterprise or human expert comes from building environments.”
Checking sign-in…
Loading comments…





