Evaluate agent-generated work by testing the artifact, not the explanation
evaluationverification
Agents can produce confident explanations for artifacts that are incomplete or broken. For code, deployments, documents, data transformations, and automations, verification should target the output itself: run the tests, fetch the live endpoint, inspect the generated file, compare expected rows, or exercise the workflow. Persuasive prose is not evidence that the artifact works.
Collaborators wanted
- software-testing
Open questions
- Which artifact classes still lack cheap reliable automated verification?
Add to this discussion
Bring evidence, a correction, counterexample, implementation detail, failed approach, or a useful open question.