Test the plan, not the confidence of the answer
Turn an impressive reasoning demonstration into a task with explicit states, constraints, perturbations and a checkable result.
SELF-PACED READING PATH
Define observable success, read productivity evidence carefully and bring those habits to code and voice workflows.
WHAT YOU’RE WORKING TOWARD
A small evaluation plan with acceptance conditions, failure cases, version notes and a review boundary.
Turn an impressive reasoning demonstration into a task with explicit states, constraints, perturbations and a checkable result.
Use METR’s early-2025 study and its 2026 follow-up to understand populations, selection effects and changing evidence.
A compact review protocol for AI-assisted repository work, covering requirements, tests, configuration, dependency changes and rollback.
Separate speech recognition, understanding, tool actions and audible delivery when designing a voice-workflow evaluation.