Building a Monitoring Agent That Does Not Cry Wolf

A monitoring agent has one job that sounds simple: watch something, and say something when it matters. In practice almost every one we have seen fails in the same direction. Not by missing something important, but by surfacing so much that the important thing drowns in it.

The failure is not technical in the usual sense. The polling works, the diff works, the alert fires exactly as designed. The system just never learned the difference between a change and a change worth a person’s attention, and after the third irrelevant alert in a week, nobody reads the fourth one either.

Watching something is the easy half

Most data sources give you one of three ways in: a webhook that fires on change, an API you poll on a schedule, or nothing at all, which means scraping a page and diffing it against the last version you saw.

Webhooks are the easy case and the rare one. Most of the sources worth monitoring, a government registry, a competitor’s site, a planning authority’s weekly export, offer no push mechanism at all, which means the system has to ask on a schedule and notice what changed. That part is genuinely mechanical: fetch, normalise, compare against the last known state, record what is new. It is also where most monitoring projects stop, because it is the part that demos well. Something changed, an alert fired, everyone is satisfied for about a week.

The actual problem is relevance

A registry that updates weekly might have thousands of new records in a single pull. A competitor’s site might change a dozen things nobody cares about for every one thing that matters. If every change becomes an alert, the system has technically done its job and functionally failed at it, because the person receiving the alerts will tune them out long before the one that mattered arrives.

Relevance has to be scored, not assumed. For a geographic monitor, that might mean distance from a fixed point, so a planning application three streets away gets surfaced and one across the city does not. For a competitive monitor, it might mean matching against a defined set of pages or keywords rather than the whole site. For a regulatory feed, it might mean filtering to the specific categories a business actually operates in. None of this is exotic engineering. It is a deliberate filter, built before the first alert ever goes out, based on what the person on the other end actually needs to act on.

Not every match deserves the same urgency

Once relevance is scored, routing still matters. A high-confidence match on something time-sensitive belongs in an immediate notification. A lower-confidence one, or something with no real deadline attached, belongs in a digest a person checks once a day rather than an interruption they get once an hour. Treating every match identically is a smaller version of the same mistake as not filtering at all: it trains the recipient to stop trusting the channel, just on a longer timeline.

This is also where a monitoring agent starts to resemble the document pipelines covered in automating document processing. The same principle holds in both: confident and clean goes through one path, and anything uncertain goes through a slower, more visible one, rather than forcing every case through a single undifferentiated channel.

The failure mode nobody builds for

The most damaging way a monitoring system breaks is silence: the source itself changes shape, and the monitor never notices that it stopped working.

A government portal redesigns its export format. A competitor migrates their site to a new framework and the page structure the scraper depended on no longer exists. The monitor keeps running, keeps polling, and simply stops finding anything, because the thing it was looking for is gone, not because nothing changed. From the outside this looks identical to a genuinely quiet week. Nobody gets an alert, because the system was built to alert on matches, not on its own failure to find any.

The fix is to monitor the monitor. A minimum expected volume over a given window, an alert if that volume drops to zero unexpectedly, and a periodic sanity check that the source still returns the shape of data the parser expects. This is unglamorous work that never shows up in a demo, and it is the difference between a monitoring system that degrades loudly, which is recoverable, and one that degrades silently, which usually gets discovered by a customer asking why they never heard about something the system should have caught weeks ago.

What good looks like in practice

A monitoring agent that works well is, from the outside, almost boring. It sends rarely. What it does send is worth reading every time. When it goes quiet, that silence is meaningful because the system has already proven it would speak up if there were something to say, and the one time it truly cannot find anything, it says that explicitly rather than staying quiet by default.

That reliability is not the product of a cleverer detection algorithm. It comes from treating relevance scoring, routing, and self-monitoring as first-class parts of the build, the same way Zapier, n8n, or custom treats durable execution as a first-class requirement rather than something bolted on once the happy path already works.

We build monitoring systems people actually trust enough to stop checking manually. See how we approach automation, or read about the pipeline architecture that makes routing like this possible.