Build now or wait for a better model: split the work into bets and assets

A&A perspective

Whether to build now or wait for the next model is not decided by how much scaffolding you have written, nor by how fast models are improving. It is decided by one thing: when the capability gap your scaffolding fills is finally closed, will you notice? There are three steps. First, take each feature you are about to build and sort it into a bet that becomes unnecessary if the model improves, or an asset that remains either way. Second, for the items you labelled bets only, write down one condition — before you start, in prose — that will tell you the gap has closed. Third, write down at the same time what you will do when that condition is met: discard the work, thin it out, or leave it in place. That is this article's answer.

A&A perspective

The reason to sort this way is that a model update does not destroy your scaffolding. It closes the capability gap your scaffolding was filling. Once the gap closes, the pre-processing and post-processing sitting on top of it have finished their job. If the gap never closes, that work stays necessary for years. So the real loss is not in having built something, but in carrying it after it has finished its job, without noticing. The only scaffolding that becomes debt is scaffolding with no means of detection. Put the other way round: write one detection condition and the loss from building on the bet side acquires a ceiling. What follows is the evidence for this judgement, and the one-page sheet to fill in before you start.

A&A perspective

What transfers here is the fact that the founder of a small team states publicly, and without hedging, where his own build work belongs. A policy of staying in the application layer only holds together if you assume the models will get better. When they do, part of the application layer becomes unnecessary — but the application layer can also be rebuilt on top of the better model. So "when models improve, my work disappears" is a less accurate reading than "when models improve, the contents of the application layer get swapped out." That is A&A's reading, not the page's claim. The same page names what the team is investing in next: "creating an interface for writers to compare versions, collaborate with AI, and navigate complex writing projects with ease." In this article’s terms those sit on the asset side — work the vendor will not do for you however much the models improve. That one sentence is where the policy of staying in the application layer becomes concrete about what you keep holding yourself.

Anthropic ↗

A&A perspective

The page does not, however, prove this policy correct. It does not say which parts of Lex's application layer were expected to survive model improvement. The three numbers it carries — signups, churn and cost — come with no population, period, definition or independent verification, and the page itself notes “This case study was written with help from Lex!” These are self-reported figures on the vendor's own site, and no causal link is shown between the division of labour and those numbers. This article does not use any of them as evidence. It uses exactly one thing: that a named founder of a small company states this division of labour openly.

Anthropic ↗

A&A perspective

The important thing in that passage is not whether to bet, but the ordering: the bet and its detection are built at the same time. The evaluation is written first and the feature catches up later. The bet is not something you wait on until it pays; it is something you place after attaching an instrument that stops you from holding it once it has lost. What a solo or two-person team cannot copy is the scale of the evaluation infrastructure. The ordering does not depend on scale. Writing one line of condition before you start works in a business with no evaluation infrastructure at all.

A&A perspective

There is an asymmetry not to overlook. The party making that statement is Anthropic, which develops the models. Being able to hold "models will get better in a few months" as a premise is not the same position as running a small business on top of those models. The vendor holds both sides of the bet; a small operator holds one. The same sentence therefore carries a different expected value depending on who reads it. What this article recommends is not placing more bets, but attaching an exit condition to the ones you place.

A&A perspective

Three questions are enough. First: is this scaffolding filling a shortfall in the model's capability, or something else? Second: is that shortfall the kind of thing the model vendor wants to close? Third: if it does close, can you remove the scaffolding, or has it grown into the customer's operations in a way that cannot be unpicked? If the answer to the first and second is "the model's side" in both cases, it is a bet. If either answer is "something other than the model", it is an asset. The third question checks whether an item you labelled a bet can later be removed safely; if it cannot, decide how you will remove it before you start. The bet-and-asset split, the content of these three questions and the asymmetry of attaching a condition only to the bets are all A&A design proposals. Neither of the sources cited below states them.

A&A perspective

The second question carries the weight. The gaps a model vendor wants to close are general capabilities that pay off across many users: holding long context, following instructions, keeping output format stable, reasoning further. Pre-processing and post-processing that exist to patch those are bets. Gaps like where a customer's data lives, an internal approval rule, the layout of a counterparty's form, an industry turn of phrase, a permission boundary — the vendor has no motive to close those. Scaffolding that fills them survives the model getting better. Those are the assets.

A&A perspective

This distinction has an explicit failure condition. Where the vendor has no reason to close the gap — a customer's private data, their approval rules, a counterparty's form layout — the work is an asset, and attaching a detection condition to it is pure overhead. The more instruments you add that never fire, the harder it becomes to know which instrument to read. Attach conditions only to the items you labelled bets, and attach none to the assets. Holding that asymmetry is what makes a single-page sheet work.

A&A perspective

One line per bet is enough, but there is a constraint on how to write it. "When the output gets better" cannot be judged. What can be judged is whether, with that scaffolding removed, the result the customer was supposed to receive still holds. In the example above, the condition for (1), the splitting step, is: pass the longest attachment through without splitting it, and see whether the resulting invoice data matches the original document on both the amounts and the number of line items. For (2), the normalisation step: remove the post-processing and see whether the output is in a format the customer's accounting software ingests as-is. Neither is an opinion about quality. Both are written as the presence or absence of a state left on the customer’s side. How to write a pass criterion as a state the customer receives is covered in detail in “When to switch models in a small AI SaaS: customer failures, not benchmarks”, so it is not repeated here. What is new here is only that the same way of writing is used for a bet’s exit condition rather than for a pass criterion.

A&A perspective

In a one-person business you need no evaluation infrastructure to run that line. What you need is to keep one or two inputs that actually broke in the past. Run those inputs once through a path with the scaffolding removed, and check by eye whether the state you wrote down holds. Three bets means three lines, and each condition should be scoped so that checking it is one run-and-eyeball. It is at the point where you start building an "evaluation suite" that you run into the cost problem covered further down.

A&A perspective

That transition carries straight over to this decision. The day the detection condition attached to a bet is met for the first time is the day that gap closed. From then on, the same condition changes role: it stops asking "can the model do this yet?" and starts asking "is the model still doing this?" The point of the structure is that the moment a bet turns into an asset arrives as an observation rather than a forecast. You do not need to guess which scaffolding will become unnecessary. You only need a mechanism that tells you when a guess came in.

A&A perspective

In practice it runs like this. When the condition is met, carry out the action you wrote down before starting: remove or thin the scaffolding on the bet side. Then re-point the same condition at confirming that the path without it has not broken. What disappeared is the scaffolding, not the knowledge the scaffolding produced. What remains is a single line — "if this input does not produce this result, something is wrong" — which is cheaper than the original build and stays usable when the model changes again.

A&A perspective

Translated into a one-person business, that becomes a concrete inspection step. When the model has gone through two generations and your detection condition has still never been met, the conclusion is not "the gap has not closed yet." Suspect the condition itself first. Has a requirement unrelated to the bet crept into it? If the condition for removing the splitting step also requires mapping the customer's own account codes, then a piece of the asset side has been mixed into a bet's condition, and no amount of model improvement will satisfy it. A mixed condition locks in the loss on that bet permanently.

A&A perspective

The inspection is simple. Re-read the condition and sort every requirement in that sentence into "the model's shortfall" or "the customer's circumstances." If even one requirement is on the customer's side, take that part out of the condition. If what remains is only things that should be satisfied once the model improves, the condition is usable. This inspection is work to finish before you add another bet.

A&A perspective

First, where the number of bets grows until maintaining detection is heavy. The method works while the sheet fits on one page; once there are ten or twenty conditions, the checking itself is a fixed cost. The moment you start thinking about automating the evaluation, suspect that the cost of detection may have overtaken the losses it prevents.

A&A perspective

Second, where you want to predict which scaffolding will become unnecessary. This method offers no prediction. Neither source gives a way to tell which parts of an application layer will be erased by model improvement. What can be designed is detection, and nothing here estimates the hit rate of bets or the development cost saved. If the decision needs a forecast number, this method is not enough.

A&A perspective

Third, where every gap is on the customer's side. If all of the scaffolding fills customer-specific data, rules and formats, there are no bets at all, and writing detection conditions is wasted work outright. The correct decision in that case is to build now, and the hesitation is coming from somewhere else.

A&A perspective

Once the bet-and-asset split is in place, the remaining work is turning the build scope into an estimate. Carving out the first unit of work to sell is covered in "Choosing your first paid AI service: how a solo founder carves out one sellable unit of work", and the whole path from winning customers to retaining them in "AI-native GTM: a practical guide for solo founders and small teams". Deciding whether to actually move to a new model once a condition has been met is a separate problem: "When to switch models in a small AI SaaS: customer failures, not benchmarks" covers building representative tasks out of customer failures. This article stops one step before that, at whether to build now at all.