The authors argue LLMs now generate research ideas faster than they can be judged, and that existing evaluators rely on parametric model knowledge or unstructured retrieval rather than experience-groundingTying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.Full definition → reasoning.
They release DeepInstruct, a dataset of controlled pairwise idea comparisons along three dimensions: novelty, significance and feasibility.
DeepInstructor improved Hit@1 and Hit@2 alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition → with human judgments by 24.4% and 29.7% over existing baselines, with the retrieved evidence making each evaluation traceable.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
grounding — Tying a model's answers to checkable sources — retrieved documents, live data, tool results — instead of letting it answer from memory alone.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
Why it matters
Demonstrates a concrete pattern for grounding an LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition →AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →'s subjective judgments in structured retrieved experience rather than raw model opinion, improving alignment with human judgments by double digits.
articleLabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI AgentsLei Liu, Yikun Zhang, Jialin Chen, Wanjia Zhao, Rex Ying, Wengong Jin, Hua Xu, James Zou, Tianyu Liu, Hongyu Zhao
articleXAI-Arena: Can LLMs Assess the Quality of XAI Explanations?Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel
articleVibe Patenting: Evaluating LLM Judges for Professional Patent-Drafting AgentsToshiaki Koike-Akino, Vlad Blaykhman, Ye Wang, Jing Liu, Gene V. Vinokur