Plan, divide, and conquer: How weak models excel at long context tasks

Splitting a long input helps only if each worker returns what the final answer actually needs. Together AI's divide-and-conquer research illustrates this with finding the second-smallest number in a list. Asking each worker for its local second-smallest number can discard the global answer; returning each chunk's two smallest numbers preserves enough information to combine the results. The research explanation makes aggregation design central.

This approach balances shorter, easier inputs against information lost between chunks and mistakes when combining answers. The authors report gains on tested retrieval, question-answering and summarization tasks. Tasks requiring tightly connected evidence across distant passages can favor an unsplit input. The useful design question is therefore what information must survive each worker's response.

“The Intuition: Don't ask one genius to read a library in an hour. Ask ten interns to read one book each.”

Together AI

“Model Noise (The "Brain Fog"): Models don't just forget linearly; they get overwhelmed. Our research shows that model confusion grows superlinearly with input length. It is mathematically better to split the task because you reset that confusion counter with every new chunk in a shorter length”

Together AI

“If you naively ask workers to "Find the 2nd smallest number" in their specific chunk, the final manager will fail because the global 2nd smallest number might be the smallest number in a specific chunk.”

Together AI

“We found that testing just 5 random samples is sufficient to find the optimal chunk size due to the predictability of the noise curve.”

Together AI

“If your task requires tracking a subtle clue from Page 1 that connects to Page 100 (like the "Dialogue Character Inference" task in our paper), the "Divide" step breaks the necessary context.”

Together AI