Settings leave no history: decide where the record of each change lives
Check whether each admin-console setting keeps a history you can roll back to, and record only the ones that do not.
Key points
- List every setting that bypasses code
- Check if each one keeps a history
- Record date, person, old value, rollback
- Ask about history before you adopt
Overview diagram in preparation
Contents
List the settings that change what the customer gets
A&A perspective
A setting that changes what the customer gets without touching code is one you can change from an admin console alone, and the answer or the feature set the customer receives changes with it.
A&A perspective
Once you rebuild how the company runs around AI, part of what reaches the customer is decided by the settings of the tools you use rather than by the code you wrote. Which model gets called, how many calls a day go through, which customer gets which feature: all three are changed from an admin console.
A&A perspective
A customer says the answers changed since last month, and you walk back through the repository without finding the change. You are not searching badly. The place where it changed sits outside the repository.
A&A perspective
The same holds for a system a contractor built for you. If the handover leaves you able to change settings yourself, the record of what you changed does not live in the delivered work.
A&A perspective
Automate & Augment (A&A), which writes this article, takes on the work of rebuilding how a company runs around AI. A&A reads this situation not as an overuse of settings, but as a place for the record that has not been decided yet.
From the sources
Cloudflare, which provides a service that relays AI calls, publishes documentation for a feature that composes those call routes from an admin console. The documentation says you can iterate without touching application code: you publish a new route version and you are done.
From the sources
The same documentation lists the parts you can place inside a route. Among them are a conditional branch, a split by percentage, a limit on the number of requests and a limit on cost.
A&A perspective
In other words, the judgement of which customer request goes to which model can sit outside the code.
From the sources
Stripe, which provides payment and billing, also publishes documentation for a mechanism that assigns the features each plan can use. That documentation says you can launch, change and experiment with your pricing without needing to change your codebase.
A&A perspective
They are different features at different companies, yet the two documents make the same promise: you can change this without touching code.
A&A perspective
What A&A recommends first is not a ban but a list. Write the settings out in four columns: the name of the setting, what changes for the customer, whether a history remains, and how to go back.
Hypothetical example
Suppose you run a service that returns draft replies to inquiries. The list would hold the model routing, the daily call limit and the features opened per plan.
Check, setting by setting, whether a history remains
A&A perspective
Once the list exists, check setting by setting whether a history of changes remains. History here means that the state before the change is kept as a copy you can return to.
From the sources
The route documentation above explains this point directly. Each change produces a new draft, it says.
From the sources
The same documentation also covers what happens after you go live. A deployed route can be rolled back instantly, and you can always roll back to earlier versions, it says.
A&A perspective
For this setting, the record of what you changed stays inside the setting itself.
From the sources
The plan-feature documentation says something else. The lookup key given to a feature cannot be edited after the feature is created, it says.
From the sources
The same documentation also describes the operation that takes a feature you no longer use out of the list. A feature you have archived cannot be unarchived, it says.
From the sources
It also says that an archived feature still creates entitlements if it is attached to existing products.
A&A perspective
This document contains neither a mechanism that keeps a change as a copy nor a procedure for returning to a previous state. Because that rests on what the text does not say, it is A&A's own reading.
A&A perspective
The place to check is the part of the product documentation that describes history or the operation that returns you to a previous state. Some documents never use the word history, so search for draft, deploy and audit as well.
A&A perspective
From outside, the two look the same. Both are changed from an admin console and both take effect fast. The difference shows up only when you try to go back.
A&A perspective
There is a third shape. Who changed it remains, but there is no procedure for returning to the previous state. Treat that case not as a setting with a history but as a setting you cannot roll back.
Hypothetical example
In the list above, the model-routing row would say that a history remains and that you can go back to a published route. The plan-feature row would say that no history is described and that going back means rebuilding it.
| What to check | Setting that composes AI call routes | Setting that assigns plan features |
|---|---|---|
| Does a change leave a copy | Each change produces a new draft | No copy is described |
| Can you go back at once | A deployed route can be rolled back instantly | No procedure for going back is described |
| When does it take effect | At the moment it is deployed | Existing subscriptions at the next billing period |
| Are there operations you cannot undo | None described | Editing a lookup key, and unarchiving a feature |
| What you record yourself | One line: the reason | Four lines, plus the reason |
Add a four-line record only where no history remains
A&A perspective
The four-line record is added only to settings where no history remains. Keeping the same four lines twice for a setting that already has a history leaves you unsure which one is right.
A&A perspective
The four lines are the time of the change, the person who made it, the value before it, and how to go back.
A&A perspective
The person who made the change writes it on the spot. Written up later in a batch, the previous value is no longer remembered.
A&A perspective
Of the four lines, the only one you cannot recover afterwards is the previous value. The time and the person can sometimes be picked up from another record, but an overwritten value is left nowhere.
A&A perspective
Where it lives does not matter. Next to the setting or in the daily work log is fine. What you decide is the single point of where it will always be.
A&A perspective
There is one more line beyond the four: why it was changed. That line is added to settings that do keep a history as well.
A&A perspective
Neither document addresses a mechanism for keeping the reason for a change. A copy keeps the previous state, not the reason for it.
A&A perspective
For settings where the day it reaches the customer shifts, write that date too.
From the sources
The plan-feature documentation notes that for existing subscriptions, product feature changes create active entitlements at the start of the next billing period.
A&A perspective
The day you changed it and the day it changed on the customer's screen are not the same. With only the day of the change in the record, you cannot line it up against the date of the inquiry.
A&A perspective
How to match the features you sell against subscription state is covered in a separate article, "Paid plans in an AI SaaS: 3 steps to match subscriptions and features". What this article adds is where the record of changing that setting lives.
Hypothetical example
You might write it like this: 12 October 14:00 / changed by me / previous value 200 calls a day / to go back, type 200 into the same screen. To that you add the reason line, "for one customer's bulk import". Where the effective day shifts, add that date as well. This is a hypothetical example.
Make the history question a condition before you start using a setting
A&A perspective
Everything so far concerns settings you already have. For settings you are about to use, write the conditions out before you start.
A&A perspective
There are three to write out: whether a history remains, whether there are operations you cannot undo, and what configuration it assumes.
From the sources
The route documentation states its prerequisites too. It accepts only one request shape, and other formats return a 400 error, it says.
From the sources
The plan-feature documentation opens by assuming that you are already using the provider's subscriptions and customer resources.
A&A perspective
You write the conditions first because history cannot be added afterwards. Finding out after you have started that there is no history does not bring the past changes back.
A&A perspective
When a contractor builds it, put these three in the request for the estimate. If the answer is that it can be changed from a setting, ask further whether that setting keeps a history.
A&A perspective
Check where in the documentation the answer you get is written. A promise that is not in the product documentation can change at the next update.
A&A perspective
There is another way to decide: stop changing settings at all and push every customer-visible change through a change to the code. The history then sits in one place.
A&A perspective
These features exist, however, to let you avoid touching code. Give that up and you give up the speed with it.
A&A perspective
An external service's admin console cannot always be bound by your own rules alone. So A&A divides by whether a history remains, rather than banning.
A&A perspective
On the day you decide to use a new setting, add its row to the list. If it is a row with no history, start the record that same day.
Summary
A&A perspective
List the settings that change what the customer gets without touching code.
A&A perspective
Check, setting by setting, whether a history of changes remains and whether you can go back.
A&A perspective
Only where no history remains, keep the time, the person, the previous value and the way back.
A&A perspective
Before you start using a new setting, write out whether a history remains and which operations cannot be undone.
A&A perspective
Once the rows of the list are filled in, the time spent searching again at every inquiry becomes the time it takes to read one table.
List the settings that change what the customer gets without touching code. Check, setting by setting, whether a history of changes remains and whether you can go back. Only where no history remains, keep the time, the person, the previous value and the way back. Before you start using a new setting, write out whether a history remains and which operations cannot be undone.
FAQ
If we stop changing settings and push everything through code, do we still need this record?
An external service's admin console cannot always be bound by your own rules alone. The records you stop needing are those for settings you can genuinely decide nobody will touch in that console. The rest stay as rows in the list.
Do settings that keep a history need nothing written at all?
They need one line: why it was changed. Neither of the two documents read here addresses a mechanism for keeping the reason. What the copy keeps is the previous state, not the reason for it.
Which tool should hold the record?
This article recommends no particular product. What you decide is the single point of where it lives; next to the setting or in the daily work log is fine. It is enough that you know where it will always be.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Dynamic routing
Cloudflare · 2026-10-02 (last updated, stated on the page)
Accessed 2026-10-26 - Entitlements
Stripe · undated documentation page; no visible publication date
Accessed 2026-10-26