PlanningBench puts LLM planning under automated verification

PlanningBench tests whether a language model can assemble a complete plan that satisfies several requirements at once. The benchmark announced by Tencent pairs each text problem with verification checklists covering constraints and objectives, so a plan that reads plausibly can still fail the checklist. Dataset documentation.

Problems are built by a generation framework over planning families, task types and constraint categories, then passed through automatic filtering and human quality review. The announcement describes the framework as usable for evaluation and training, but the dataset card says the 467 synthetic instances released are intended as test data, not training data. Construction and release scope.