Ask a team that rolled out an AIOps platform last year whether it worked and you will usually hear a confident yes, followed by a pause when you ask for the numbers. The honest reason for the pause is simple. The tool went live, the dashboards changed, and nobody had saved a record of how operations looked on the day before. Without that record, “it feels calmer” is the best evidence anyone can offer, and a feeling is not something a finance lead will renew a contract on.

This piece looks at where the AIOps field stands on that problem and why a baseline captured first has quietly become the dividing line between teams that can defend their results and teams that cannot.

The state of AIOps claims

Vendors in the AIOps space promise faster diagnosis, fewer pages and smarter correlation across logs, metrics and traces. Many of those promises are plausible. The weakness is on the buyer’s side: very few organisations arrive at the pilot with a documented picture of their current operations. So when results are reported, they are compared against memory, and memory is generous to whoever made the purchase.

The pattern is easy to recognise in post-launch reviews. One person says incidents are resolved quicker. Another says the on-call rota feels lighter. A third points out that a quiet quarter would have produced the same impression. Each statement could be true, and none can be tested, because there is no earlier measurement to hold up beside the new one.

What a baseline actually contains

A baseline is not a grand study. It is a short, dated note written before anything changes, covering three kinds of information.

• What it looks like. A plain description of the current routine: which tools raise alerts, who receives them, how an incident moves from first signal to closure, and where people say they lose time.
• What it costs. The recurring spend on monitoring and incident tooling, plus a rough count of the hours people put into triage in a typical week.
• How long it takes. The elapsed time between a problem appearing and someone acting on it, and between acting and closing it, taken from the ticket history you already have.

Notice what is missing: no target, no promise, no opinion about the tool. A baseline records the starting position and nothing else. The moment aspirations leak into it, it stops being a measurement and turns into a goal document.

Why the “before” picture decays quickly

Operational reality changes under your feet. Teams reorganise, services are retired, a seasonal peak passes. If you wait until the pilot is half finished to reconstruct the earlier state, you are reconstructing it from tickets that were closed under different conditions. That is why the capture has to happen before the first configuration change, not alongside it.

A sensible window is a few ordinary weeks of normal work, long enough to include a bad day and a quiet one. Save the export, note the date, and write down anything unusual that happened, such as a major outage or a holiday freeze. Those footnotes later explain odd data points that would otherwise be blamed on the new platform.

A practical sequence for the pilot team

1. Pick the two or three questions the pilot is meant to answer, in ordinary words.
2. For each question, find the existing record that speaks to it, whether that is the ticketing export or the paging log.
3. Freeze a copy of those records with a date stamp, stored where it cannot be tidied away.
4. Write the qualitative notes: how the rota feels, which alerts people ignore, which steps are done by hand.
5. Only then switch on the first integration.

If you want a companion read on the same operational territory, there is a discussion of how smarter tooling is meant to quiet the flood of IT events, and a look at where machine assistance fits into IT decision making. Both read better once you know what your own starting figures were.

A parallel from planning abroad

Anyone weighing a costly change against how things stood before it faces the same discipline. Take people who travel overseas for eye care: some start their homework on BestEyeHospitals, which collects directories and planning pages for patients coming from other countries. Its medical disclaimer is explicit that the pages are general reading, are no replacement for a clinician’s advice, and must not become an excuse for putting care off. Operations teams can borrow the humility: a reference page orients you, but your own documented facts carry the decision.

Reading the results honestly

After the pilot, return to the frozen records and ask the same questions in the same way. Compare like with like: the same ticket categories, the same hours, the same definition of “resolved”. If the picture has improved, you can say so and point at the evidence. If it has not, you have learned something just as valuable at a fraction of the cost of a full rollout.

There is a quieter benefit as well. Teams that keep a baseline become better at spotting drift. When a metric moves for reasons unrelated to any tool, the stored starting point stops people from crediting the wrong cause. Over a few cycles that habit improves every later decision, from renewing licences to deciding which service deserves automation next.

What to take from the trend

The AIOps market will keep producing impressive demonstrations. The teams that benefit most are not the ones who buy fastest, but the ones who can show, in their own words and their own figures, where they began. Capture that picture first. It costs a few days of careful note taking, and it is the only thing that lets anyone, including you, say with a straight face whether the change was worth it.