Six months on from the proud launch, the system is still running and nobody has thought about it since. You ask the room who checks whether it is still making good decisions. There is a pause. A few people look at each other. The honest answer is no one. The thing went live, everyone moved on, and the question of who keeps it healthy never landed on a desk. That gap is where the cost builds up, out of sight, until something visible breaks.
What you are actually deciding before launch
You are deciding who holds the system once the launch buzz fades. A deployed AI system keeps living after go-live, and a live thing needs an owner, a check rhythm, and a plan for when it needs refreshing. The choice is easy to skip. Either one named person is accountable for keeping it healthy, or it becomes an orphan that degrades while everyone assumes it is fine.
The reason this matters more for AI than for ordinary software is that AI does not hold still. A model depends on the data it was built on, so when the real world it sees in production moves away from that, its performance slips. A study from researchers at MIT, Harvard, and the University of Monterrey found that 91 per cent of machine learning models degrade in performance over time, an effect the authors call AI aging. They saw not just rising error rates but growing variability, so behaviour becomes erratic rather than steadily worse. The dangerous version is the system that still looks fine while making slightly worse calls every month.
So the real decision is who monitors it, and how lightly they can do that without building a department. The rest comes down to three calls. What does an owner actually watch. How do you decide what to fix and what to log. And who is the right person to hold it, because the obvious candidate is often the wrong one.
When light-touch oversight is the right call
Light-touch oversight is right for the systems that touch customers or money but do not need a data science function to supervise. The owner’s job here is supervision, not maths, so they never touch a statistical test. They make sure someone watches three things, and that someone raises a hand when one of them looks wrong. That is enough to catch drift early, before it does real damage.
Those three things map onto how monitoring works in practice. The first is the data going in, whether the information feeding the system still looks like what it was built on. The second is the decisions coming out, sampled and sense-checked against what actually happened. The third is whether the thing is even running properly, speed, errors, and availability. A non-technical owner can ask good questions about all three without touching a line of code.
One trick keeps this sustainable. Instead of asking a person to check every output, route only the low-confidence ones to a human. The system flags where it is unsure, a person reviews those, and the routine decisions flow through untouched. That keeps a human in the loop on the cases that matter without turning oversight into a full-time job nobody has the hours for.
When you need a heavier review rhythm
You need a heavier rhythm when the system carries real risk, makes decisions at scale, or sits under regulation, because silent drift there is no longer just an inconvenience. A monthly look replaces a quarterly one, the data and a fresh sample of outputs get examined properly, and the review gets written down. The weight of oversight should match the weight of what the system decides.
The EU AI Act offers a useful gold standard even where your system is not legally classed as high risk. For high-risk systems it requires a risk management process running across the whole lifecycle, automatic event logs kept for traceability, and accuracy and security maintained throughout the system’s life rather than signed off once at launch. You do not have to be regulated to borrow the shape of that. Logs, a lifecycle view, and accountability that does not expire are sound practice for anything important.
This is also where documentation earns its place. A system that several people rely on, with no record of what it was built on or when it was last reviewed, is one staff change away from becoming an orphan. A short written note for each important system, what it does, what data it uses, who owns it, and when it was last checked, is cheap to keep and saves a scramble later.
What it costs to get this wrong
Getting it wrong is expensive precisely because the failure is invisible until it is not. An unowned system does not announce its decline. It keeps producing credible-looking output while the decisions behind it slowly degrade, which is far more dangerous than an obvious crash because nothing prompts anyone to look. By the time a wrong answer surfaces in front of a customer or in the numbers, the system may have been drifting for months.
Borrow the way finance people think about debt to keep the upkeep cost honest. MIT Sloan Management Review frames AI technical debt like financial debt, with four parts. There is the principal, the cost of updating the outmoded thing. There is the interest, the running cost of working around it. There are the liabilities, the extra problems caused by leaving it. And there is the opportunity cost, what you lose by doing nothing. The point is not to drive debt to zero, it is to manage it deliberately.
That gives a non-technical leader a clean way to triage. Sort each issue by how bad the debt is and how much it affects the business. Fix the high-debt, high-impact ones now. Schedule the high-debt, low-impact ones before they grow. Stay alert to the low-debt, high-impact ones. And for the low-debt, low-impact ones, just log them and move on. You are spending attention where it returns the most, not chasing a perfect system.
Who should actually own it, and what to ask
The right owner is an existing leader who will still be looking at the system in six months, which is exactly why it should rarely be the founder. Founders drive a launch, then their attention moves to the next idea, and the unglamorous monitoring falls through the floor. Large firms appoint a Chief AI Officer. An owner-managed business needs minimum viable accountability instead, ownership handed to someone already in post, clarity over organisational purity.
So pick a person whose role naturally touches the system and who is not about to be pulled onto something else. Give them the three-layer watch, a review slot in the diary that matches the system’s importance, and the authority to escalate to whoever built it. Write their name next to the system on a one-page list. That list, more than any policy document, is what stops orphans forming, because it makes the absence of an owner impossible to hide.
If you are weighing this up for the systems already live in your business, the most useful first move is to make that list and put names against it. Where a name is missing, you have found an orphan. If you would value a second pair of eyes on which systems matter most and how lightly you can sensibly watch them, book a conversation.
