Change the labels, keep the planning problem

What the historical paper asks

Noam Brown’s archived post points to the preliminary o1 PlanBench evaluation. That paper compares familiar Blocksworld descriptions with obfuscated and randomized variants, alongside other conditions. It reports different outcomes across representations. This is evidence about the paper’s specific setup and historical models, rather than proof of how every reasoning system works or performs today.

Construct an equivalent pair

Create a tiny scheduling puzzle with explicit prerequisites, a starting state and one goal. Write a valid solution yourself. Make a second version by consistently renaming objects and actions while preserving their definitions. Give each version its own answer key. Check that the renaming did not accidentally add ambiguity; an unclear rewrite tests comprehension and prompt quality as well as planning.

Score transitions separately

For each proposed solution, mark whether every action is allowed and whether the final state meets the goal. Save the earliest invalid transition instead of judging the explanation’s confidence. Compare repeated attempts under the same budget if you later run the exercise. This original toy protocol has not been executed and cannot measure general intelligence. It helps reveal precisely what a larger evaluation would need to control.

Put it to work.

• Validate the puzzle yourself.
• Rename consistently.
• Check every transition.
• Report the narrow task conditions.

Go to the source.

Original material, with its own scope and publication date. This note is an explanation, not a CooScope performance test.

1. @polynoamial — archived X post (2024-09-23)
2. Valmeekam et al. — o1 on PlanBench, version 1Link / source review: 2026-10-15