Adding AI decisions: prepare tests that show what improves
Before adding AI decisions, prepare tests that compare both versions on past work.
Key points
- Make tests from failed inquiries
- Compare versions on the same inquiries
- Weigh gains and review effort for AI
Overview diagram in preparation
Contents
Turn failed inquiries into test cases
A&A perspective
Here, a test means giving AI a past task and checking whether it produces the expected result. Before adding an automated decision, prepare a way to compare the current process with the proposed addition. If that comparison is not possible yet, prepare the tests first.
Hypothetical example
Consider a hypothetical design company that already uses AI to summarize customer inquiries. A team member reads the summary and original message, then asks the customer for information needed to prepare an estimate. The proposed addition would let AI decide what to ask.
From the sources
AI developer Anthropic recommends adding complexity only when it demonstrably improves outcomes. Its approach starts with the simplest solution and increases complexity only when necessary.
A&A perspective
Automate & Augment (A&A), the organization publishing this article, applies that principle to requests for additions to internal AI workflows. Before listing new features, list examples of current problems and what would count as an improvement.
Hypothetical example
For the design company, choose an inquiry that reached estimate preparation despite lacking a deadline. The expected result is a draft question asking about the deadline. Treat a plausible deadline invented by AI as a failure.
A&A perspective
Save the message and attachments that were available when the inquiry arrived. Including the deadline obtained later would let the system decide using information it did not have at the time. A team member separates the original materials from answers obtained afterward.
From the sources
Anthropic's evaluation guide describes starting a test collection with real failures. It also recommends including cases where a behavior should occur and cases where it should not.
A&A perspective
If every inquiry lacks information, AI could pass by always asking another question. Include inquiries that already contained everything needed and required no follow-up. Make 'no additional question needed' a valid result so the tests can expose unnecessary exchanges.
Compare both versions on the same inquiry
A&A perspective
Compare the current process and proposed addition using the same inquiries. Give the reviewer the same materials for both. If only the proposed version receives difficult cases, or only one version uses a new price list, the reason for any difference will be unclear.
Hypothetical example
In the design company's current process, AI summarizes and a person identifies missing information. With the addition, AI also identifies missing information and drafts questions, which a person checks against the originals. During the comparison, keep these questions internal and use saved inquiries rather than sending them to customers.
A&A perspective
Create one comparison row per inquiry. Record the necessary checks, the current result, the proposed result, what the person corrected and how long that took. Check both whether required information was identified and whether a question repeats something the customer already supplied.
Hypothetical example
For example, an attachment may contain the deadline while the message says only 'under the attached conditions.' If the proposed version cannot read the attachment and asks for the deadline again, that is not an improvement. This hypothetical case helps identify which materials AI must be able to read.
From the sources
Anthropic explains that AI outputs vary across runs, even on the same task. A task that passes once may fail on the next run, so the guide describes using multiple trials to examine results.
A&A perspective
A&A recommends recording successful attempts together with the total number of attempts. Keep missed information and unnecessary questions, rather than selecting favorable outputs. The number of trials needed depends on the consequences of the task and variation in results; passing a small set does not guarantee production reliability.
A&A perspective
Have the responsible team member define the criteria first. If some inquiries do not require a deadline, include those conditions. When reviewers disagree on the right answer, resolve the handling before scoring AI performance. An unclear expected result also leaves the decision about the addition unclear.
A&A perspective
Measure review time consistently, from the moment the person starts reading the original to the moment the questions are finalized. Record AI generation time and usage charges separately. If the model or reference materials also change, record those changes and do not attribute the entire improvement to the added decision step.
| What to examine | Current process | With the added decision |
|---|---|---|
| Required checks | Misses remaining after human review of the current process | Misses remaining after human review of the proposed process |
| Unnecessary questions | Questions removed during review and unnecessary questions remaining after finalization | Questions removed from AI's draft and unnecessary questions remaining after finalization |
| Human effort | Time from reading originals to finalizing questions | Time from reading originals to finalizing questions, including corrections to AI's draft |
| Ongoing cost | Current usage charges and maintenance | Added usage, test creation and updates |
Decide with improvements and review effort
A&A perspective
Decide by comparing required checks caught, new errors and human review effort. Even if the tested cases improve, consider whether the team can sustain the process when it adds too much material to read or too many corrections to make.
Hypothetical example
In the design-company example, fewer failures to flag missing deadline information, no increase in unnecessary questions and shorter review time would support proceeding. If people rewrite the questions every time, keeping the current summary-only process is also an option. These are hypothetical decision conditions, not measured results.
A&A perspective
Keep the tests even if you decline the addition. They can reveal concrete work to address first: some attachments cannot be read, or the conditions for asking about a deadline are undefined. Separate the feature request from these prerequisites when discussing the work with an implementer.
From the sources
Creating and maintaining tests also takes effort. Anthropic explains that keeping an evaluation suite useful requires ongoing attention and clear ownership.
A&A perspective
A&A proposes separating the initial cost of creating tests from the cost of updating them as the business changes, whether employees or contractors do the work. Assign who updates expected results when inquiry requirements change. Freed time is available capacity; it does not automatically become additional profit.
A&A perspective
Decide permission to send messages separately, even if the addition is adopted. This example retains human approval before sending. Add failures found in production to the tests, and assign the conditions and person responsible for reverting to the previous process. Passing the comparison alone does not authorize automatic sending.
From the sources
For AI systems that direct their own process, Anthropic also describes using stopping conditions, such as limits on iterations, to maintain control.
A&A perspective
A runtime limit and an owner's decision to decline an addition are different decisions. One limits execution; the other weighs cost and benefit. The source's design guidance is useful, but whether the addition is worthwhile for your company depends on your own comparison results and maintenance burden.
Tests make the reason for an addition explainable
A&A perspective
Turn past failed inquiries and previously successful cases into tests.
A&A perspective
Compare both versions using the same materials, recording missed information, unnecessary questions and human correction time.
A&A perspective
Use improvements, review effort and test maintenance to decide whether to add the new step or retain the current process.
Turn past failed inquiries and previously successful cases into tests. Compare both versions using the same materials, recording missed information, unnecessary questions and human correction time. Use improvements, review effort and test maintenance to decide whether to add the new step or retain the current process.
FAQ
How many test cases are enough?
There is no universal number. Anthropic's 20–50 tasks are a starting point for early evaluation, not a sufficiency threshold for an operating workflow. Start preparing with a small set of failures, add routine and exceptional cases, and expand testing according to the consequences of the work.
Do we need a dedicated evaluation tool?
An initial comparison can be organized in a spreadsheet. First connect input materials, expected results, both versions' outputs and human corrections. Consider automated testing when the number of cases or frequency of updates makes manual comparison burdensome.
Can we try the addition before building tests?
A trial on reversible internal drafts with human review is an option. To use that trial as evidence for production adoption, record how it differs from the current process. If all you have is a one-off impression, the addition's effect remains unverified.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Building effective agents
Anthropic · 2024-12-19
Accessed 2026-10-31 - Demystifying evals for AI agents
Anthropic · 2026-01-09
Accessed 2026-10-31