Designing AI monitoring alerts: page a person only when they can act
Which events page a person at once is decided by whether the receiver can act, not by severity.
Key points
- Page a person only if they can act now
- Events with no move go to the batch
- List exceptions that stay immediate
- Put the tiers in acceptance criteria
Overview diagram in preparation
Contents
Call a person at once only when they can act
A&A perspective
A tier is a group of alerts sorted by when they get read. The immediate tier carries only events the person who receives them can act on right away. Having a move means the action to take is already decided at the moment the alert lands.
A&A perspective
Sorting by severity leaves you nothing to do but raise and lower a threshold. Send the heaviest events first and the receiving side keeps reading alerts that say "this is bad and nothing can be done now". Once skimming becomes the habit, the alerts that do have a move scroll past the same way.
From the sources
Gumloop, a US company that sells workflow automation, published an account of its own support operation on 17 February 2026. The article says its support team is only two people.
From the sources
According to that article, the agent that watches platform health alerts a human if, and only if, a finding is actionable. The article also describes the point of always-on monitoring as keeping the team informed without adding noise.
A&A perspective
Automate & Augment (A&A), the company writing this article, does not read that condition as a trick for cutting alert volume. We read it as the definition of what qualifies for the immediate tier. Volume is something a threshold can change. Which events go out at once is something only the entry condition can change.
A&A perspective
When you deliver AI under contract, A&A thinks one more question comes first: who is the person with the move. When you monitor a client's system, the client is often the only side that can change a setting, and your side can only report.
Hypothetical example
Say the AI you delivered fails while waiting on an external service. If re-running it fixes the problem, the move belongs to your side. If the client's own plan limit caused it, the move belongs to the client. The same failure changes who gets called at once.
Everything else drops into the daily and monthly batch
A&A perspective
Not calling someone at once is not the same as pretending the event did not happen. Events with no move are not discarded; they move into a batch with a fixed reading time.
From the sources
The same automation company's article describes tiers separate from the immediate alert. Every 24 hours an agent fetches the last 24 hours of support tickets and posts a formatted summary to the team chat.
From the sources
Every month, a different agent reviews the past 30 daily summaries. From those it extracts the most common error patterns. The article says the resulting report is compared against the list of issues not yet worked on, and the agent suggests which ones should be created to address them.
A&A perspective
A&A reads this not as three strengths of alert but as three tiers with different reading times. Immediate is the tier for acting on a single event. Daily is the tier for seeing the shape of one day. Monthly is the tier for finding what a single event cannot show.
A&A perspective
Once the tiers are separate, every event has somewhere to land. A failure with no move shows up as a count inside the daily summary. A count that keeps climbing shows up as a pattern in the monthly re-read. An event that fits no tier is an event with no reason to be recorded at all.
Hypothetical example
Say a response delay happens a few times a month. Sent immediately, the receiver decides to wait and see every single time. Seen as a count in the daily summary, the month it rises sharply is the first month anyone needs to act.
A&A perspective
A&A thinks each item in the daily batch needs columns beyond its subject line. Add a column for which user hit it. Add another for how many times the same cause has come back. Without those columns the daily summary stays a list of counts and the monthly re-read cannot extract a pattern.
A&A perspective
That said, the company that published this operation monitors its own product. When you monitor a client's system under contract, whether the daily summary also goes to the client becomes a separate question. If it does, the hour the client reads it belongs in the acceptance criteria too.
List up front what stays immediate even without a move
A&A perspective
The actionability test needs exceptions: events whose cost of being missed grows with time.
A&A perspective
A&A treats a safety-related stop as something to log and alert at once even when no move exists at that moment. A statutory filing deadline gets the same treatment. So does any process that sends money out of the account. The time until someone can act is itself the size of the loss.
A&A perspective
This exception is not a condition stated in the sources. Screen purely on whether a move exists and the events whose loss grows with every hour of delay fall into the daily batch.
From the sources
Anthropic, a US company that develops AI, published a developer explanation of how to build agents on 19 December 2024. It says agents can pause at predetermined points and wait for human judgement. It also says they can pause when they hit a blocker.
From the sources
The same explanation says it is common to include stopping conditions to maintain control. A maximum number of iterations is given as an example. The page itself carries a note saying that much of the tooling landscape it describes has changed since it was published.
A&A perspective
A&A reads the exceptions as something to list up front, the same way the pause points are decided up front. Add them after operations begin and the reason for each addition is lost, so nothing carries into the next delivery.
Hypothetical example
Say a refund looks like it ran twice. Even with no way to stop it on the spot, it belongs in the immediate tier. Wait a day and the number of duplicated refunds grows by that much.
A&A perspective
A&A uses these questions to pick the exceptions. If you notice a day late, does the loss grow? If it grows, does it grow in proportion to time? Can you shorten the wait until someone with a move appears?
Write the three tiers into the acceptance criteria
A&A perspective
The three tiers are not something to settle once operations have started. They go in before delivery, as part of the acceptance criteria. Leave them out and no one owns the miss when a miss happens.
A&A perspective
A&A's table has five columns: the event name, the tier it goes in, the recipient to notify, the response deadline carried only by the immediate rows, and what happens if that deadline slips. Those five columns fit the whole alert promise on one sheet.
Hypothetical example
For a delivered billing AI, it reads like this. A failure while waiting on an external service goes in the immediate tier. The recipient is the delivery side on call, and the response deadline is 30 minutes within business hours.
Hypothetical example
A stop caused by a user entering the wrong input goes in the daily tier, with the client's own staff as recipient. This is a worked example; the real deadline follows the client's working hours.
From the sources
The agent explanation lists several conditions for the tasks agents suit. Clear success criteria are one. Being able to feed the result of an attempt back into the next move is another. Meaningful human oversight is listed as a condition as well.
A&A perspective
A&A reads that "meaningful human oversight" as the alert table itself. Oversight that never says who looks at what, when, and what they do with it keeps only the shape of oversight.
A&A perspective
With the table in place, cutting alerts and tolerating misses become different conversations. A proposal to shorten the immediate rows is a proposal showing those events have no move. Dropping an event that does have a move is a proposal to tolerate a miss, and that needs the client's agreement.
A&A perspective
A&A has not measured any relationship between having this table and the rate of missed events. This is a design argument about how to set the tiers, not a forecast of results.
Summary
A&A perspective
Call a person at once only for events the receiver can act on right then. Events with no move drop into the batch: the daily summary and the monthly re-read.
A&A perspective
Events whose loss grows with time stay immediate even without a move, and they get listed up front. Keep the three tiers and the exceptions as an acceptance-criteria table written before delivery.
Which events page a person at once is decided not by severity but by whether the receiver can act right then. Events with no move drop into the daily and monthly batch, only events whose loss grows with time become exceptions, and the three tiers stay in the acceptance criteria written before delivery.
FAQ
If we cut the immediate alerts, will we miss more?
Events taken out of the immediate tier still get read in the daily summary and the monthly re-read. Only the reading time changes; nothing leaves the record. The only events that may disappear are the ones with no reason to be in any tier.
What if the client says they want every anomaly straight away?
Make everything immediate and the client stops reading it. So show them the response deadline attached to the immediate rows. Once it is clear that only so many rows can hold a deadline, narrowing the list becomes a quality conversation.
So we do not have to set threshold numbers?
The numbers are still needed. What comes first is the tier. Once the tier is fixed, the number that puts an event in the immediate tier is set at the level where a move actually exists.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Supporting the world's most AI-native companies with a 2-person team
Gumloop · 2026-02-17
Accessed 2026-10-24 - Building effective agents
Anthropic · 2024-12-19
Accessed 2026-10-24