What do LLMs think when you don't tell them what to think about?

Minimal prompts can reveal recurring patterns in a language model's output that task-specific tests rarely exercise. Together AI's researchers removed chat templates and supplied topic-neutral beginnings or punctuation, then categorized the resulting text. Different model families repeatedly favored different subjects and formats across the tested prompts and labeling methods. Their research account describes the experiment.

For example, OpenAI's GPT-OSS frequently generated programming and mathematics, while Qwen often produced multiple-choice questions. These are patterns in generated text under a particular setup. They offer an additional way to audit model behavior, without establishing private thoughts, topic expertise or what a model would produce in its normal assistant interface.

“Despite the absence of explicit task or topic specification, LLMs generate diverse content; however, each model family exhibits distinct topical preferences . GPT-OSS favors programming and math, Llama leans literary, DeepSeek often produces religious content, and Qwen tends toward multiple-choice questions.”

Together AI

“We use topic-neutral, open-ended seed prompts such as “Actually,” “Let’s think step by step,” or even just punctuation like “.” We remove chat templates entirely—no system prompt, no roles—and use standard decoding.”

Together AI

“GPT-OSS overwhelmingly defaults to programming (27.1%) and mathematics (24.6%). More than half of a model family’s output concentrates in these two domains!”

Together AI

“We find that GPT-OSS frequently produces advanced or expert-level content (68.2%) , such as depth-first search, breadth-first search, or dynamic programming (see Figure 3). Llama and Qwen skew much more toward basic or intermediate material.”

Together AI

“Degenerate text turns out to be one of the clearest windows into safety and privacy risks, precisely because it reflects uncontrolled generation . These behaviors rarely appear in standard benchmarks, yet they are highly revealing.”

Together AI