Guide
A real citation can still support the wrong claim
Audit a short AI-generated answer at the claim, passage and source-family levels. A working link is only the beginning.
Sources & evidenceResearch workflow
Source-based4 min ↗
A CONNECTED RESEARCH AREA
Evaluation & trust
Look beyond a score or a convincing answer. Inspect methods, source support, failure modes and the limits of human oversight.
Benchmarks & methodsReliability & failuresSources & evidenceSecurity & permissionsHuman review
Start exploring.
Source-based reading in this area.
Research note
Read the update before repeating the productivity number
Use METR’s early-2025 study and its 2026 follow-up to understand populations, selection effects and changing evidence.
Benchmarks & methodsEngineering practice
Source-based4 min ↗
Research note
Read agent reliability beyond a single successful run
A short route into τ-bench’s final-state and repeated-trial ideas, with a practical way to record a tiny task suite.
Agent reliabilityRepeated trials
Source-based4 min ↗
Checklist
Evaluate the monitor as well as the agent
Use SHADE-Arena as a research example for asking what a transcript monitor can see, miss and falsely flag.
MonitoringEvaluation design
Source-based4 min ↗
Workflow
Give preference and correctness separate scorecards
A compact review workflow for comparing AI answers without letting warmth, verbosity or visual polish stand in for task success.
Human preferenceReview rubrics
Source-based4 min ↗