When customer AI usage spikes: split caps into halt and bill lines
Before a spike, set two AI usage caps per customer: a halt line for accidents and a bill line for added usage.
Key points
- Split the AI cap in two: halt and bill
- Halt line counts calls; set who unblocks
- Bill line counts spend; notify, invoice
- Act by the line hit: investigate or bill
Contents
Split the AI cap in two: halt and bill lines
A&A perspective
If your service has AI built in, give each customer a usage cap made of two separate lines. A usage cap here means the value that decides how far one customer can use your AI features.
A&A perspective
The first is the halt line. It stops costs from ballooning after an accident, such as a bug that repeats the same call, or leaked login details that let someone else use the account. The second is the bill line. It charges, on top of the monthly fee, for usage a customer legitimately added as their own work grew.
A&A perspective
The reason to split them is that you pay for the AI first. An API, the connection your service uses to call an outside AI model, charges for every call. When a customer's usage suddenly jumps, that month's cost leaves your account before you can invoice the customer.
A&A perspective
A single cap gets one of the two cases wrong. Set it low, and you halt a growing customer as if it were an accident. Set it high, and the cost of an accident piles up until billing day.
From the sources
Cloudflare offers a feature called AI Gateway that logs the calls an AI app makes and can limit its usage. Its documentation gives two purposes for rate limiting: preventing expensive bills and preventing suspicious activity.
A&A perspective
Automate & Augment (A&A), the company writing this article, gives each purpose its own line, because that makes it easier to decide what happens after a line is hit.
A&A perspective
Suspicious usage stays halted until you have checked it. Usage that grew legitimately is billed, and the customer keeps working. One line cannot do both.
Halt line counts calls in a time window
A&A perspective
Set the halt line as a number of calls per period of time, because accidents tend to show up as calls bunched into a short time. Decide, together with the line, who restores a halted customer and when.
From the sources
In Cloudflare's documentation, the limit is the number of requests sent within a set period; its example is 100 requests per 60 seconds. A request over the limit gets a 429 response, meaning too many requests, and is not processed.
From the sources
You can also choose how the period is counted: a fixed window that follows the clock, or a sliding window that counts the most recent stretch of time. The documentation's example is a limit of 10 requests per 10 minutes, with 10 requests sent at 12:09 and 10 more at 12:11. Under a fixed window all 20 succeed; under a sliding window, the requests sent at 12:11 push the previous 10 minutes over 10, so they fail.
From the sources
The documentation also says that the default limit set in the Cloudflare dashboard applies uniformly to every request through that AI Gateway.
A&A perspective
From this, A&A reads four things to decide for the halt line: what to count, which window to count in, what the customer sees on screen when they go over, and who restores a halted customer.
A&A perspective
The one most often missed is who restores. A customer using the service legitimately can hit the halt line just by starting a new kind of work. Once halted, they cannot use it until you check and lift the halt. If they are halted on Friday night and nobody looks until Monday morning, that customer has an outage all weekend.
A&A perspective
So set the height of the halt line to match the hours your team can act. If you can only respond during weekday business hours, set the line higher. Write down in advance who gets the alert when the line is hit, and how to lift it.
A&A perspective
If you want a different line for each customer, also check that the system holding your halt line supports per-customer limits. Left at the default, every customer gets the same line.
| What to decide | Halt line | Bill line |
|---|---|---|
| What the line is for | Stop costs ballooning from accidents or suspicious use | Turn legitimately added usage into revenue |
| What it counts | Calls per period of time | The amount used in a month |
| What the customer sees when it is hit | The request is not processed and an error comes back | They keep working and get a notice and an invoice |
| Who restores a halted customer | Only you | Not needed: it does not halt |
| Easy to forget | How the time window is counted | An expiry on extra allowance you give |
Bill line counts spend for the month
A&A perspective
Set the bill line as the amount a customer used in a month. Going over it does not halt the customer: you let them know and add the overage to the invoice. Legitimate growth shows in the monthly total, not in the number of calls in a short burst.
From the sources
Stripe, which provides online payments and billing, documents a setting called billing thresholds. It can issue an invoice partway through a billing period, as soon as a customer's usage reaches a set amount. The stated aim is to limit the amount owed or the usage that builds up between invoices.
From the sources
Stripe also says it bills usage after the fact. There is an alert that reaches the provider's system when a customer's usage goes past a set level, including a type that fires only once per customer, the first time they go over. The alert is in public preview, a stage before general release.
A&A perspective
What A&A watches for is that this alert reaches the provider, not the customer. What the customer sees is up to you. Tell customers they are near the cap and what the overage rate is, and let them choose whether to continue; then nobody on your side has to make the call in the middle of the night.
From the sources
Stripe has a mechanism called credits: a usage allowance given to customers, prepaid or free. Credits do not expire unless the expiry field is set. They can only be applied to metered items, the invoice lines charged by the amount used.
A&A perspective
If you give a spiking customer extra allowance, A&A's view is to set its expiry at the moment you hand it over. Forget, and the same allowance keeps working next month.
A&A perspective
If you sell only a flat monthly fee, the invoice has no line to carry an overage. With nowhere to put a bill line, set only the halt line and talk to the customer when they hit it. Whether to add a metered line to the contract can be decided after that.
When usage spikes, act by the line hit
A&A perspective
If you set both lines before a spike, handling the spike comes down to which line was hit. There are four cases.
A&A perspective
If only the halt line was hit, the monthly total still has room, but the number of calls in a short time is abnormal. Investigate it as an accident. If only the bill line was hit, the short-term call count is normal and only the total has grown. Notify the customer and bill.
A&A perspective
If both were hit, halt first and then investigate. If neither was hit, do nothing. A judgment you would otherwise struggle with mid-spike becomes a matter of seeing which of the four cases applies.
Hypothetical example
What follows is a hypothetical example, not figures from a real company or anything A&A has measured. Say a four-person company sells, for a monthly fee, a service with AI built in that drafts replies to inquiry emails from clients. It pays for the AI per call, up front.
Hypothetical example
The halt line is calls per hour per customer, set at several times that customer's usual busiest hour. Above it, requests are not processed, the customer's screen asks them to try again later, and the company's staff get an alert. A staff member checks and lifts the halt by hand.
Hypothetical example
The bill line sits where the month's usage passes what the base fee includes. Going over does not halt the customer; the company notifies them and adds the overage to the invoice at the unit rate. Any extra allowance expires at the end of that month.
A&A perspective
This article does not set the values themselves. Where the halt line belongs depends on how your service is normally used. Line up one month of records, each customer's busiest hour and monthly total, and you will have a rough idea where both lines go.
A&A perspective
A&A has not measured the effect of splitting the cap. The sources are all the providers' own product documentation, not studies comparing ways to set caps. If your AI cost is small next to the monthly fee, a single halt line takes less work.
Summary
A&A perspective
Split the AI usage cap into two: a halt line that stops accidents and a bill line that charges for added usage. Set the halt line as calls per period of time, and decide who restores a halted customer.
A&A perspective
Set the bill line as the monthly amount; going over does not halt the customer, so notify them and add it to the invoice. When usage spikes, act by the line hit: investigate if it is the halt line, bill if it is the bill line.
Decide the lines only after a spike and you end up weighing halt against bill on the spot. Start with your heaviest customer: write down both lines' values and what that customer sees when crossing each.
FAQ
If we sell only a flat monthly fee, do we still split the cap in two?
No. The invoice has no line for an overage, so set only the halt line and talk to the customer when they hit it. Whether to add a metered line to the contract can be decided after that.
How do we set the halt line's first value?
Look at one month of records and set it above that customer's usual busiest hour. This article does not recommend a specific value, because it depends on how your service is used.
Do free-trial users get the same two lines?
No. Users who are not paying yet have nowhere for a bill line to go. How free allowances end is covered in “AI service free trials by days or usage: a cost cap and 3 endings”.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Rate limiting
Cloudflare · Last updated Sep 30, 2026
Accessed 2026-10-23 - Cloudflare AI Gateway overview
Cloudflare · undated documentation page
Accessed 2026-10-23 - Set up thresholds
Stripe · undated documentation page
Accessed 2026-10-23 - Set up usage-based alerts
Stripe · undated documentation page
Accessed 2026-10-23 - Billing credits
Stripe · undated documentation page
Accessed 2026-10-23