Research note
Check who decides that a robot succeeded
A technical review note for separating task execution, control architecture and the labels used to judge a recorded attempt.
THE SHORT ANSWER
A successful-looking rollout and a trustworthy success label require different evidence.
Separate the components
Lucy Cai’s archived Instance announcement proposes a success detector for robot rollouts. Its accuracy and latency comparisons remain unverified publisher claims here. As a separate architectural reference, Reachy Mini’s documentation places hardware I/O, safety checks and sensors in a daemon, with an SDK client above it. Execution architecture and outcome judging therefore deserve separate review questions.
Define visible success
For an offline review exercise, choose a harmless recorded or simulated task and write its completion rule. List the evidence needed to apply it, such as the object’s final location, a complete action sequence or an unobstructed view. Allow “cannot tell” when the recording lacks that evidence. A missing view should not silently become either a successful or failed attempt.
Review the judge’s mistakes
Have a person label a small set containing obvious success, obvious failure and deceptive near-success. Compare any proposed automatic labels against those references, keeping false success separate from false failure. Check disagreements in the human labels before trusting the comparison. This is an original evaluation outline, not an Instance test or robotics safety procedure. No robot was connected, and no rollout media or physical behavior was reviewed.
Put it to work.
- Define completion explicitly.
- Allow insufficient evidence.
- Include deceptive near-success.
- Separate false success and false failure.
Go to the source.
Original material, with its own scope and publication date. This note is an explanation, not a CooScope performance test.
Source published · Editorial review
Archived English post text; primary sources reviewed separately. Generated annotations, media and missing replies are outside the evidence scope.
- @lucyjcai — archived X post (2026-07-13)
- Hugging Face — Reachy Mini core concepts and architectureLink / source review: 2026-10-15