When work comes back 'not quite right': set acceptance with two samples
Acceptance for work judged on tone is set by two samples the client picks, not by a worded standard.
Key points
- Write acceptance as two chosen samples
- Offer three to five; the client picks
- Put the choosing time in the estimate
- Set the review month, then pick again
Contents
Adjectives have nothing to compare against; two samples do
A&A perspective
Work judged on tone and voice cannot have its acceptance written as a number or a rule. Whether it passes is decided by how it reads.
A&A perspective
Automate & Augment (A&A), who writes this article, designs acceptance criteria for AI-made deliverables and builds the systems behind them.
A&A perspective
Seen from there, one mistake comes up again and again. Trying to write the acceptance condition, you keep adding adjectives.
A&A perspective
Natural tone. Not too stiff. Close to the reader. The more precise the words, the longer the condition grows, and the rejections keep coming. No adjective gives the review anything to compare against. Something to compare against means a real document you put side by side when you decide whether the work passes.
From the sources
A case study published by Anthropic describes Lex, a company that builds writing software in which AI returns advice to the writer. After experimenting with various large language models, Lex chose Claude as their primary AI model for its tone, cost, and quality of output.
From the sources
The founder calls Claude like a sage editor. Its tone and personality give its answers a smarter, more human vibe, and it feels less bullet point-y and less robot-generated, he says.
From the sources
The founder also calls choosing a model like casting the right actor. The voice of the model he picked matched the brand and the user experience the company wanted to create. The same account says the company focuses on how the advice impacts users emotionally.
A&A perspective
This is a case study the model provider published about its own product, not an outside audit. What A&A looks at here is the order of the reasons. What came first was not a capability score but tone.
A&A perspective
Where you look to decide has moved too. Not a property of the output itself, but how the person who read it felt. You cannot redefine something decided by the receiver in the words of the sender. So the acceptance condition holds a real document to compare against, not adjectives.
A&A perspective
Two of them, to be exact. One accepted sample and one rejected sample. The client picks both. The rejected one is there to show the wrong direction. An accepted sample on its own does not tell you which way counts as a miss.
Hypothetical example
Say you deliver a monthly report to a client. The accepted sample is last month's report they wrote themselves; the rejected one is a report from elsewhere that they felt was off. With those two, the reason for a rejection turns into "here is where it differs from the accepted one".
You supply the candidates; the client picks
A&A perspective
The client picks the samples. If you declare what passes, the thing the review compares against is back inside the supplier's own words.
From the sources
Anthropic published an article on January 9, 2026 explaining how to evaluate AI agents, meaning AI that works through a given job while choosing its own steps. The article splits grading into three kinds: code-based graders, model-based graders, and human graders.
From the sources
That article lists five methods for letting a model grade: rubric-based scoring, natural language assertions, pairwise comparison, reference-based evaluation, and multi-judge consensus.
A&A perspective
What the list shows is not a ranking. Writing the standard in words and comparing against a reference sit in the same list as working options.
From the sources
The same article also lists the weaknesses of model-based grading: it is non-deterministic, it is more expensive than code, and it requires calibration with human graders for accuracy. Among the strengths of human graders, it says they are used to calibrate model-based graders.
A&A perspective
A&A applies that calibration to contract work. The question is who counts as the human grader there. Even with the judgement automated, a step where a person decides pass or fail remains. In contract work that person has to be the client. A verdict you issue yourself does not hold at the review.
A&A perspective
So the procedure runs like this. You prepare the candidates, and which becomes the sample is the client's call.
A&A perspective
Offer three to five. With one, nothing has been picked. Past five, the client stops picking and replies "make them all good".
A&A perspective
Offer the same number of rejected candidates. Not ones you built to miss on purpose, but real documents. A rejected sample you wrote yourself cannot leave the range you already imagined.
Hypothetical example
Say you line up three accepted candidates from the client's own past material and three rejected candidates from documents published elsewhere. All the client does is circle one on each side. It fits inside a single meeting.
| What to look at | Written as adjectives | Written as two samples |
|---|---|---|
| End of revisions | Until they are satisfied | Until it matches the accepted sample |
| Where the end point sits | Inside their feel | In two documents both sides see |
| Readable before you start | No | Yes |
| How the estimate reads | Unlimited, or padded high | Revision rounds priced as hours |
| If they have no samples | Unchanged | Bound rounds and scope instead |
Put the time spent choosing samples into the estimate
A&A perspective
The time spent choosing samples goes into the estimate as work done before you start. It is the review brought forward, so it belongs with the hours spent making the deliverable.
A&A perspective
Fold it into a free meeting and two things happen. The client does not pick seriously. And you lose your reason to hold off starting until they have.
A&A perspective
The gain from putting it in the estimate is not a bigger number. It is that the end of the revisions is fixed while you are still estimating.
A&A perspective
Under a condition written as adjectives, revisions run until they are satisfied. The end point sits inside their feel, so you cannot read how many rounds it takes before you start. What you cannot read, you cannot price. The supplier either takes unlimited rounds or pads the number and loses the job.
A&A perspective
With two samples, the end point becomes the distance to the accepted one. A distance is something both sides can look at together, so the number of revision rounds can itself be written as hours.
Hypothetical example
Say you include two rounds of revision in the estimate and bill separately from the third. You can bound the number because the accepted sample points at what the third round would fix.
A&A perspective
There is one case where this design does not hold. A project where the client has no samples to give.
A&A perspective
A new line of business may have no past material. Some clients will not spend the time to pick. In those cases, forcing a sample into being does not help: the person who picked it is not convinced, so it does not hold at the review.
A&A perspective
Then you drop quality-as-it-reads from acceptance altogether. Rather than argue pass or fail, bound the estimate by revision rounds and scope; it comes out cheaper for both sides. You stop trying to define quality in words at all.
A&A perspective
The other way round, the same two samples work when you are the one receiving the work. Instead of telling a contractor to make it more natural, hand over the accepted and the rejected sample, and the revisions go back and forth less.
Set the review month up front and pick the samples again that month
A&A perspective
Samples go stale. So you set the review month at the same time as you fix them.
A&A perspective
When the client's business changes, their readers change. When the readers change, last year's accepted sample can become this year's rejected one.
A&A perspective
Carry fixed samples into a long maintenance term and the roles swap. The sample you put there to stop rejections becomes the reason for one. "I built it exactly like the sample" meets "that is not the sample any more" at the review.
A&A perspective
There is one way to prevent it. Write the review month in the same field as the acceptance condition. Put it in another table or another memo and it gets forgotten, busiest month first.
A&A perspective
Four lines in that field are enough. Where the accepted sample lives, where the rejected one lives, the date they picked, and the review month. Those four lines replace the paragraph you used to write in adjectives.
Hypothetical example
Say it is an ongoing contract: write the review month as once every six months, up front. When that month comes, you line up new candidates and the client picks one on each side again. Even if they pick the same ones, the date is refreshed, so the next six months run on a current sample.
A&A perspective
The meeting to pick again is shorter than the first one. There is already something to compare against, so the client only has to hold the previous accepted sample against the new candidates.
A&A perspective
Acceptance for AI-made deliverables also raises a separate question: how to write down the fact that the same input does not produce the same result. That one is covered in a different article, "Acceptance criteria for AI deliverables that vary from run to run". This article is about changing the words of pass and fail from adjectives to real documents.
Summary
A&A perspective
Acceptance for work judged on tone and voice is not settled by making the words more precise. It is settled when the client picks an accepted and a rejected sample and you fix those two as what the work is compared against.
A&A perspective
You line up three to five candidates and the client picks. A verdict you decide yourself does not hold at the review.
A&A perspective
The time spent choosing goes into the estimate as work done before you start. The end of the revisions is fixed while you estimate, so the rounds can be written as hours.
A&A perspective
Write the review month in the same field as the acceptance condition. When samples go stale, the sample itself becomes the reason for a rejection.
Acceptance for work judged on tone and voice is not settled by making the words more precise. It is settled when the client picks an accepted and a rejected sample and you fix those two as what the work is compared against. You line up three to five candidates and the client picks; a verdict you decide yourself does not hold at the review. The time spent choosing goes into the estimate as work done before you start, so the end of the revisions is fixed while you estimate and the rounds can be written as hours. Write the review month in the same field as the acceptance condition, because a stale sample becomes the reason for a rejection. Where the client has no samples to give, drop quality-as-it-reads from acceptance and bound the work by revision rounds and scope.
FAQ
What do I do when the client says "any of them is fine" and will not pick?
Do not decide for them; drop quality-as-it-reads from acceptance instead. A sample the client is not convinced by does not hold as something to compare against at the review. Rather than argue pass or fail, bound the estimate by revision rounds and scope. With the choosing time already a line in the estimate, the refusal itself tells you which of the two routes this project takes.
Can the accepted sample be a piece I made myself and am proud of?
Put your own work in as the accepted sample and the thing being compared against is back inside your own words. The client is the one who passes or fails the work at the review, so unless they picked it themselves, a rejection cannot be restated as a distance from the accepted sample. Offer your piece as one of the candidates they choose from.
With two samples, will revisions always end in two rounds?
No. What the samples fix is not the number of rounds but where the end point sits. Because the end point becomes the distance to the accepted sample, you can estimate the number of rounds before you start. The number itself is still set per project. Work drafted with AI varies in tone from run to run; how much of that variation passes is also read as a distance from the accepted sample.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Lex streamlines the writing process with Claude
Anthropic · undated case-study page; no publication or update date shown
Accessed 2026-10-22 - Demystifying evals for AI agents
Anthropic · January 9, 2026
Accessed 2026-10-22