What the check interval actually tells you
The interval is the distance between two checks. It is not the time to the alert — and precisely this confusion sits behind most gut decisions. An outage does not follow your check schedule. It begins somewhere between two checks, and to a first approximation without any preference for a particular moment.
From this follows a simple calculation. Assume the start of an outage is uniformly distributed between two checks. In the best case the service fails immediately before the next check — the delay approaches zero. In the worst case it fails immediately after a successful check, and a full interval n passes before the next check even looks. Averaged over many incidents you land at n/2, and in the worst case at n. For five minutes that is 2.5 minutes on average and 5 minutes in the worst case; for one minute, 30 seconds and 60 seconds.
What matters is what this value describes: the moment of the first failed check, not the alert. In any serious setup, confirmation still sits between the two — and it costs further intervals. Anyone who configures "five minutes" and reads it as "alerted within five minutes at the latest" still has the larger part of the calculation ahead of them.
The assumption of uniformly distributed failure times holds for random disturbances — hardware, network, resource exhaustion. It holds less well when outages correlate with operations: deployments, cron jobs on the hour or load peaks at known times cluster at particular points on the clock. If your monitoring runs in the same cadence as the cause, checks and disturbances can systematically miss each other. That is an argument for offset check times, not against the calculation.
The calculation: interval plus confirmation rounds
Hardly any monitoring system alerts on the first failure — and rightly so. A dropped packet, a TCP reset or a briefly overloaded firewall is enough for a single down without any user ever noticing. This is why monitoring systems require several consecutive failures before they report. Why the noise becomes unbearable without this filter is spelled out in the article on taming flapping alerts.
This confirmation is not free; it is paid for in time. Let n be the interval and c the number of consecutive failures required. The first failed check arrives after n/2 on average, and every further confirmation costs a full interval. This yields two compact formulas:
- Mean detection = n/2 + (c − 1) × n — the value you may expect across many incidents.
- Worst-case detection = c × n — the value you should use in your SLO calculations.
- Plus timeout: A hanging server only counts as down once the timeout expires. This value adds to every round and is missing from most gut estimates.
Plugging in common combinations produces a picture that surprises many. The share refers to the error budget of roughly 43 minutes that an SLO of 99.9 percent leaves over 30 days (Google SRE Workbook, Chapter 5) — the derivation follows in the next section.
| Interval and confirmations | Mean detection | Worst case | Share of the 43-minute budget |
|---|---|---|---|
| 1 minute, 1 check (no confirmation) | 0.5 min | 1 min | 2.3 % |
| 1 minute, 2 confirmations | 1.5 min | 2 min | 4.6 % |
| 1 minute, 3 confirmations | 2.5 min | 3 min | 6.9 % |
| 3 minutes, 2 confirmations | 4.5 min | 6 min | 13.9 % |
| 3 minutes, 3 confirmations | 7.5 min | 9 min | 20.8 % |
| 5 minutes, 2 confirmations | 7.5 min | 10 min | 23.1 % |
| 5 minutes, 3 confirmations | 12.5 min | 15 min | 34.7 % |
The row where most discussions turn is the second to last. A five-minute interval with two confirmations — a perfectly common default — detects after 7.5 minutes on average and 10 minutes in the worst case. That is double what the number "5" suggests, and it is only the beginning of the recovery time: notification, acknowledgement, diagnosis and remediation come on top.
The interval is the first summand of detection time, not its result. Anyone who sets an interval without counting the confirmation rounds and the timeout is planning with a number that is too small by a factor of two to three. The honest entry in the runbook is not "5-minute monitoring" but "worst-case detection: 10 minutes plus timeout".
The blind spot: outages that fit between the checks
So far the question has been how late an outage is detected. The more uncomfortable question is whether it is detected at all. Sampling has a structural property: events shorter than the sampling distance can fall through the grid entirely. The SRE Book describes the same effect for metrics — observing CPU load over the span of a minute will not reveal even quite long-lived spikes that drive high tail latencies (Google SRE Book, Chapter 6). Chapter 4 likewise warns that a measurement averaged over a window can hide an instantaneous load twice as large as the average one (Google SRE Book, Chapter 4).
For uptime checks this can be quantified exactly. An outage of duration d, positioned randomly within the check grid, is hit by a check with interval n with probability d/n, as long as d is smaller than n. If d is greater than or equal to n, at least one check falls inside the outage window — so it is captured.
| Outage duration | 1-min interval | 3-min interval | 5-min interval |
|---|---|---|---|
| 30 seconds | 50 % | 17 % | 10 % |
| 90 seconds | captured | 50 % | 30 % |
| 3 minutes | captured | captured | 60 % |
| 6 minutes | captured | captured | captured |
Now confirmation enters the picture — and considerably sharpens the blind spot. For c consecutive checks spaced n apart to all fail, the outage has to cover at least the span between the first and the last of them. From this follows a hard lower bound: an outage is only reportable once it lasts longer than (c − 1) × n.
- 1 minute, 2 confirmations: outages under 1 minute raise no alert.
- 1 minute, 3 confirmations: lower bound 2 minutes.
- 3 minutes, 2 confirmations: lower bound 3 minutes.
- 5 minutes, 2 confirmations: a 4-minute outage cannot be reported at all — it slips between two checks.
- 5 minutes, 3 confirmations: lower bound 10 minutes. Everything below stays invisible to alerting.
A four-minute outage during the lunchtime peak can cost more orders than half an hour at night. If your configuration can structurally neither report such incidents nor show them in the history, your uptime statistic is systematically too optimistic. That is not a measurement error but a property of the chosen parameters — and the reason why availability figures from different setups are only comparable to a limited degree. How the percentages translate into time is shown in the article on calculating and understanding uptime SLAs.
Cross-checking against the error budget
Up to here these are arithmetic exercises. They only turn into a decision once you hold them against the error budget. The Site Reliability Workbook states the relationship unambiguously: with an SLO of 99.9 percent over a time window of 30 days, a constant 0.1 percent error rate uses exactly all of the error budget — a burn rate of 1 (Google SRE Workbook, Chapter 5). The same reference table works out the extreme case: a burn rate of 1,000, meaning a 100 percent error rate, exhausts the budget in 43 minutes (Google SRE Workbook, Chapter 5). That is the entire room for manoeuvre that 99.9 percent leaves in a month.
Against this background, the table from the second section reads differently. A single incident detected with a five-minute interval and two confirmations in the worst case consumes roughly 23 percent of the monthly budget for detection alone. With three confirmations it is close to 35 percent. Two such incidents in a month and the budget is spent before anyone has started remediating.
The cross-check becomes even clearer when you place the design point of the reference alerts next to it. For a 99.9 percent SLO the workbook recommends a page alert at a burn rate of 14.4 over a one-hour window, corresponding to 2 percent budget consumed at alert time; a second page tier at burn rate 6 over 6 hours corresponds to 5 percent, and a ticket at burn rate 1 over 3 days to 10 percent (Google SRE Workbook, Chapter 5). Two percent of 43.2 minutes is roughly 0.9 minutes. The alert is therefore designed to fire after about a minute of total outage.
If the alerting logic is designed around two percent of budget consumed while detection alone costs twenty-three percent, then the alerting logic is not the problem. Then the interval is the brake, and every fine adjustment of thresholds above it is cosmetics.
— Project experience
From this a usable rule of thumb can be derived: detecting a single incident should cost at most around ten percent of the monthly budget. At 99.9 percent that is about 4.3 minutes in the worst case. A one-minute interval with two confirmations (2 minutes) stays comfortably within that bound, and with three confirmations (3 minutes) it still does. A three-minute interval with two confirmations (6 minutes) already breaches it, and a five-minute interval clearly does. Anyone wanting to dig deeper into the budget logic will find it derived at length in the article on error budget and burn-rate alerts.
The rule of thumb turns with the target. At 99.5 percent roughly 216 minutes per month are available — the same ten percent then allow about 21 minutes of detection, and a five-minute interval is unremarkable. At 99.99 percent a good 4 minutes per month remain; for 99.999 percent the workbook cites the extreme value that a complete outage exhausts the monthly budget in 26 seconds (Google SRE Workbook, Chapter 5). Chapter 4 explicitly advises not to derive the target from current performance but from what the service has to deliver for its users (Google SRE Book, Chapter 4). First the SLO, then the interval.
What the one-minute cadence costs
If shorter detects better — why not set everything to the shortest possible cadence? Because the interval is also paid for on the other side of the scale, in four currencies. Remarkably, the SRE Book draws the line considerably earlier than most gut decisions do: probing for a 200 status more than once or twice a minute is probably unnecessarily frequent (Google SRE Book, Chapter 6). The same chapter notes that checking hard drive fullness for a service targeting 99.9 percent availability more than once every one to two minutes is probably unnecessary (Google SRE Book, Chapter 6), and that collecting per-second measurements of CPU load might yield interesting data but may be very expensive to collect, store and analyse (Google SRE Book, Chapter 6). On that reading, a one-minute cadence already sits at the upper end of what is sensible, not at the lower.
The arithmetic is plain: a one-minute cadence from three locations yields 3 × 1,440 = 4,320 checks per day, roughly 129,600 per month. A five-minute cadence comes to 3 × 288 = 864 per day, roughly 25,920 per month — a factor of five. For a static page this is irrelevant; for a dynamic endpoint with database access it is not.
With an equal probability of a transient error per check, the number of isolated blips grows linearly with the check rate. Five times as many checks mean five times as many opportunities for a random down. That is arithmetic, not pessimism — and the reason shorter intervals need more confirmation, not less.
The SRE Book is uncompromising here: every page should be actionable (Google SRE Book, Chapter 6). And it names the human limit openly — one can only react with a sense of urgency a few times a day before becoming fatigued (Google SRE Book, Chapter 6). Whoever fills that limit with noise no longer has it available for the real incident.
Frequent checks from a few IP addresses look like a bot. Firewalls, WAF rules and rate limits respond accordingly, and the monitor ends up measuring the defence rather than the service (Project experience). Which addresses to allow is covered in the article on firewall whitelisting for monitoring tools.
The third currency is the most expensive. The SRE Book quantifies it plainly: paging a human is a quite expensive use of an employee's time (Google SRE Book, Chapter 6). An alert that fires without being actionable lowers the value of all future alerts. Shortening the interval without thinking through the confirmation therefore buys detection time against credibility — a bad trade. How to tier thresholds and channels cleanly is shown in the article on configuring monitoring alerts correctly.
Confirmation from a second location instead of a shorter interval
This brings us to the actual point of this article. There are two fundamentally different ways to confirm an outage — and they carry completely different price tags.
- Temporal confirmation: you wait for further consecutive failures from the same check point. Price: (c − 1) × n minutes of additional detection time. What gets filtered are transient disturbances at the target.
- Spatial confirmation: several locations check in the same cadence, and an outage only counts as confirmed once several of them see it. Price: almost no additional time, because the checks run in parallel. What gets filtered are local network, routing and peering problems of the checking node.
The difference is decisive because both methods improve the same metric, but only one of them pays for it. The workbook distinguishes four attributes of an alerting strategy: precision — the proportion of events detected that were significant; recall — the proportion of significant events detected; detection time — how long it takes to send notifications; and reset time — how long alerts fire after an issue is resolved (Google SRE Workbook, Chapter 5).
Put in that language: temporal confirmation buys precision with detection time. Spatial confirmation buys precision with infrastructure — detection time remains practically untouched. So anyone who wants to reduce false alarms and only turns the counter c inevitably worsens detection time. Whoever adds a second location instead gets the precision without paying the time.
| Property | Temporal confirmation (more failures) | Spatial confirmation (location consensus) |
|---|---|---|
| Additional detection time | (c − 1) × interval | almost none, since parallel |
| Filters primarily | Transient disturbances at the target | Network, routing and peering problems of the checking node |
| Effect on precision | increases | increases |
| Effect on detection time | worsens with every round | stays practically unchanged |
| Shortest reportable outage | grows to (c − 1) × interval | unchanged |
| Detects regional disruptions | indistinguishable from a total outage | yes, visible as a partial outage |
The last row deserves attention. A single checking node fundamentally cannot distinguish whether the service has failed or whether only the path to it was disrupted. Only several perspectives turn that question into an answerable one. Why a single measuring point is structurally too little for this is explored in the article on the benefits of multi-location monitoring.
XICTRAQ builds on exactly this: reachability is checked from three distributed EU data centres in Nuremberg, Helsinki and Falkenstein, and an outage only counts as confirmed with the consensus of two of the three nodes. Because the three nodes measure in parallel, this confirmation costs no additional round. Intervals range from 5 minutes on the Free plan through 1 minute on Starter and Business to 30 seconds on the Agency plan.
The most useful order in operations is: location consensus first, then confirmation rounds, then the interval. Whoever reverses it and starts with the interval buys the most expensive improvement first and the cheapest one last.
— Project experience
A decision framework: outage cost and SLO instead of gut feeling
This finally allows the opening question to be answered cleanly — not with a number, but with a calculation. The benefit of shortening the interval is the detection time saved, valued at what a minute of outage costs:
Annual benefit ≈ outage cost per minute × detection minutes saved per incident × incidents per year. You read the saved minutes off the table in the second section; the outage cost per minute is known only to your business; the incidents per year come from your own history. Only once these three numbers are on the table does the interval question become decidable.
A worked example — explicitly with assumed values, not measured ones: a shop generates 40 euros of contribution margin per minute during peak hours and had six reportable incidents last year. Moving from 5 minutes to 1 minute, with two confirmations in each case, lowers mean detection time from 7.5 to 1.5 minutes, saving 6 minutes per incident. Six incidents × 6 minutes × 40 euros comes to roughly 1,440 euros per year. The plan's price difference has to be held against that amount — and against that alone, not against a feeling of safety. The full method behind this calculation is laid out in the article on calculating monitoring ROI.
The second test is the cross-check against the SLO from the fourth section: worst-case detection should stay noticeably below the monthly budget. Both tests together lead to a classification by service profile rather than by gut feeling.
| Service profile | Typical SLO | Outage cost per minute | Sensible combination |
|---|---|---|---|
| Information and reference site | 99.5 % | negligible | 5 min, 2 confirmations |
| Lead website with forms | 99.9 % | low to medium | 1 min, 2 confirmations |
| Shop and checkout | 99.9 % | high, varies by time of day | 1 min, 2 confirmations, location consensus |
| Payment or core API | 99.95 % and better | very high | 30 sec, 2 confirmations, location consensus |
| Internal service without customer contact | 99.5 % | indirect | 5 min, 2 confirmations, ticket instead of page |
Two values in this table are deliberately kept qualitative. Outage cost per minute cannot be stated in general terms — it depends on business model, time of day and season, and any external number here would be guesswork. The same caveat applies to the SLO: Chapter 4 of the SRE Book recommends keeping targets simple and avoiding absolutes, and recalls that availabilities of 99 and 99.999 percent are referred to as "2 nines" and "5 nines" — for Google Compute Engine it cites a target of 99.95 percent (Google SRE Book, Chapter 4). A comparison of common commitments can be found in the article on hosting uptime guarantees compared; which intervals are included in which plan is shown in the plan overview.
Setting interval, confirmation and timeout together
In practice the interval is one of three controls that belong together. Turning only one of them creates side effects on the other two. The following order has proven itself (Project experience):
- Set the SLO first. Work out the monthly budget in minutes and derive the maximum acceptable detection time from it — around ten percent of the budget per incident.
- Activate location consensus before touching the interval. It reduces false alarms without costing detection time. That is the only place in the system with this property.
- Choose confirmations deliberately. Two consecutive failures filter most isolated blips. Three are acceptable at short intervals and expensive at five minutes.
- Keep the timeout smaller than the interval. A 30-second timeout with a 30-second interval lets checks run into each other. As a guide: timeout at most one third of the interval.
- Count the timeout into detection time. A hanging server only counts as down once it expires — on every round again.
- Offset the check times. If the monitor runs exactly in the cadence of cron jobs or deployments, it systematically measures the wrong moments.
- Define maintenance windows. Planned downtime should not consume the error budget or trigger a page.
- Tier by criticality. The status page does not need the same cadence as the checkout. The interval is a control per service, not a global setting.
What comes after that decides the rest of the recovery time. Detection is only the first summand: notification, acknowledgement and escalation follow, and poorly tiered escalation can eat up every minute gained. The article on incident management and escalation covers this chain in detail. For external communication, a status page with a robust history is also worth having, so that measured availability stays verifiable.
This verifiability is increasingly no longer optional. Anyone who has to evidence availability towards clients needs a complete, dated history rather than an assertion — which records are expected across the supply chain is set out in the article on NIS2 evidence for service providers. The same operational discipline that applies here to interval and confirmation also applies to handling vulnerabilities: there too, response time is what counts, as the article on EUVD and CVE feeds shows.
Finally, classifying the measured values benefits from looking beyond the uptime check. The SRE Book summarises the observation of a service in four golden signals: latency, traffic, errors and saturation (Google SRE Book, Chapter 6). An uptime check covers a slice of that. Rising latency or saturation often announce an outage before the reachability check responds at all — which is why response time trends and the timing waterfall provide the real early warning and take pressure off the interval (Project experience).
Choosing the interval deliberately instead of escalating it
The question of 1, 3 or 5 minutes has no universally valid answer — but it does have a traceable derivation. The interval n yields detection of n/2 on average and n in the worst case, the confirmation rounds each add a full interval, the timeout comes on top, and outages below (c − 1) × n remain invisible to alerting. Against this sum stands an error budget of roughly 43 minutes per month at 99.9 percent (Google SRE Workbook, Chapter 5). Anyone who puts both numbers side by side no longer needs a gut feeling.
Just as important is the opposite direction: the one-minute cadence is not an end in itself. The SRE Book draws the line of what is sensible at once or twice a minute (Google SRE Book, Chapter 6), and every additional check creates load, noise and opportunities for false alarms. The more effective lever against false alarms lies elsewhere anyway — in the consensus of several locations, which improves precision without costing detection time.
In practice this means: set the SLO, convert the budget into minutes, choose the interval so that worst-case detection takes up a small part of it — and switch on location consensus before thinking about shorter cadences. What that feels like in practice can be seen without registration in the live demo of the dashboard; which intervals the plans cover is set out in the pricing overview.
This article draws on: Google — The Site Reliability Workbook, Chapter 5 (Alerting on SLOs) for the burn-rate reference table (burn rate 1 = 0.1 % error rate = 30 days, burn rate 10 = 1 % = 3 days, burn rate 1,000 = 100 % = 43 minutes), the relationship between 99.9 % over 30 days and a burn rate of 1, the recommended alerting parameters (14.4x over 1 hour = 2 % budget consumed, 6x over 6 hours = 5 %, 1x over 3 days = 10 % as a ticket), the short window as one twelfth of the long one, the extreme value of 26 seconds at 99.999 %, and the four attributes precision, recall, detection time and reset time. Google — Site Reliability Engineering, Chapter 6 (Monitoring Distributed Systems) for the probing frequency for a 200 status, the hard drive check at 99.9 percent targets, the cost of per-second measurements, the limits of averaging over a minute, the actionability of alerts, alert fatigue, the cost of paging humans and the four golden signals. Google — Site Reliability Engineering, Chapter 4 (Service Level Objectives) for choosing targets without deriving them from current performance, the warning about averaged measurement windows, the nines notation and the cited Google Compute Engine target of 99.95 %. In addition, our own operational experience from XICTRAQ monitoring is included, marked in the text as (Project experience). The formulas for detection time, hit probability and shortest reportable outage are derived in the text; the cost example works with explicitly assumed values and is not a survey.