Vibeleaderboard
← All Intel
Intel / video

Stanford CS329A Self-Improving AI Agents | Part 4 | Learning from Feedback with Tools/Code

Source
youtube.com
Author
Stanford Online
Date

Context

A model that reasons without checking anything can reason its way to a wrong answer. The ReAct method, covered in a Stanford CS329A lecture, has the model alternate between writing a thought and taking an action such as a search, then reading what comes back before reasoning again. prompting makes a model show intermediate steps, but those come from its own internal with no outside check. The lecturer described ReAct as prompting in language space, which avoids training a full policy.

In the lecture's example, a question about what other device can control the Apple Remote was designed for, plain prompting and chain of thought did not give a fully correct answer, while the ReAct run searched, retried a missing page under a new term, and reached an answer. The paper's abstract says ReAct overcomes made-up facts and error propagation in chain-of-thought reasoning on HotpotQA and Fever. The lecturer added that it did not always beat chain of thought on HotpotQA.

Terms in this piece · Glossary
  • chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
  • State — Information describing a system at a particular moment, such as its current text, settings or saved records.
  • Program — A set of instructions stored so that a computer can run it.
  • grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
Read the source www.youtube.com
More from Stanford Online
Recommended reads
Comments

Checking sign-in…

Loading comments…