How to Keep an Assumptions Log for an AIOps Rollout

Upgrade & Secure Your Future with DevOps, SRE, DevSecOps, MLOps!

We spend hours scrolling social media and waste money on things we forget, but won’t spend 30 minutes a day earning certifications that can change our lives.
Master in DevOps, SRE, DevSecOps & MLOps by DevOps School!

Learn from Guru Rajesh Kumar and double your salary in just one year.


Get Started Now!

Every AIOps change request carries a quiet list of beliefs. The model will see enough history. The telemetry arrives on time. Someone will read the output. None of this is written down, so nobody can test it, and the first time anyone discovers a belief was wrong is usually during an incident. An assumptions log is the habit of writing those beliefs into the ticket or design document, next to a confidence rating and a plan to check each one.

This guide is a procedure you can follow on your next change, whether that is a latency anomaly detector, an automated ticket router or a forecast of disk usage. The steps are deliberately small. A first log can be finished in under an hour, and it earns its keep the first time a result looks odd and you can open the log to see what you originally expected.

Step 1: Pick one concrete change

Do not log assumptions for “our AIOps strategy”. Pick a single change with a boundary: for instance, “flag unusual response times on the checkout service and open a ticket.” Put that sentence at the head of the document. Everything below it has to be a belief that must hold for that sentence to be true.

A narrow scope matters because vague scopes produce vague assumptions such as “data quality is good”. A narrow scope produces testable ones, such as “the checkout service emits a latency sample at least once per minute from every node.”

Step 2: Brain-dump the beliefs without judging them

Give yourself ten minutes and jot down every sentence that starts with “we assume” or “this only works if”. Include the dull ones. The dull ones are where the damage hides. Prompts that help:

  • Where does the input data come from, and how late can it be?
  • What does “normal” look like, and who decided that?
  • Which people or systems must act on the output, and are they aware of it?
  • What changes regularly in the environment (deployments, seasonal traffic, maintenance windows) that the method may not know about?

Do not edit while you write; several lines will turn out to be the same belief worded differently.

Step 3: Give each assumption a shape

A line in a log needs four fields to be useful. Use a table in your wiki, a ticket template or a plain text file; the format matters far less than the discipline.

  1. Statement: a single sentence, phrased so it could be false. “Timestamps from the load balancer and the application agree within a few seconds” is testable; “clocks are fine” is not.
  2. Confidence: high, medium or low. Resist percentages; they imply a precision you do not have.
  3. Impact if wrong: what breaks, and how visibly.
  4. Check: the cheapest way to find out, and who will do it.

The combination of low confidence and high impact tells you what to verify first. High confidence and low impact can wait, and sometimes you can leave them alone entirely. The log is not a to-do list of everything; it is a way of deciding what deserves attention.

Step 4: Turn the top items into checks

Choose the three to five assumptions with the worst combination of doubt and impact, and write a check for each. In AIOps work a check is often a query rather than a meeting. Examples that fit a latency detector:

  • Count missing samples per node over the last month, and compare with the claim that data arrives every minute.
  • Replay the last quarter of data and see how many known incidents the method would have flagged, and how many harmless days it would have flagged as well.

Record the result next to the assumption, with the date. A belief that has been checked changes status: confirmed, refuted or still unknown. Refuted beliefs are not failures of the log; they are the reason you kept it.

Step 5: Plan around what cannot be checked yet

Some assumptions can only be tested by production traffic. That is acceptable if the log says so: mark them “verify in flight” and note what you will watch. If you believe seasonal traffic will not confuse the detector, schedule a check on the first busy weekend.

Step 6: Review the log when the numbers move

Open the log on three occasions: when you ship, when results surprise you, and when the environment changes materially. Surprise is the most valuable trigger. When an alert pattern looks strange, scan the list and ask which assumption would produce it. For the day-to-day side of troubleshooting that this supports, see an introductory article on troubleshooting live IT problems. In practice a surprising result often traces back to a line that was marked “medium” confidence and never revisited.

The log also helps people who join later. A reviewer who sees a decision with no context tends to either distrust it or defend it by reflex. A reviewer who can read “we assumed the vendor feed would stay unchanged; we noted that was low confidence” can reason about whether that world still exists.

A worked example, kept short

Suppose a team wants to forecast storage growth so that capacity is added before it runs out. Their first log might read like this in prose:

  • Growth follows a steady trend rather than a step change after releases. Confidence: medium. Impact: a forecast that is far too optimistic. Check: plot the last year and mark release dates on it.
  • Retention policies are applied the same way on every volume. Confidence: low. Impact: forecasts for a few volumes would be wrong in either direction. Check: export the retention settings and diff them.
  • Procurement can add capacity within the planning window. Confidence: medium. Impact: a correct forecast that nobody can act on. Check: ask the people who order the hardware or raise the cloud quota.

Notice that two of the three lines are not about statistics at all. That is typical. AIOps projects fail on process, ownership and data plumbing far more often than on the choice of algorithm, and a log makes those risks as visible as the modelling ones.

Where outside learning fits

Writing the log is a skill, and so is judging which assumptions deserve checking. Teams sometimes look to structured training for that kind of discipline. One example of what is available is a training site based in India whose menu points to certification tracks, courses, advisory work and other offerings for people working in IT; weigh a site like that on what it says about itself and judge fit for your team separately. The log itself needs nothing beyond a text editor and agreement to use it.

Common mistakes to avoid

  • Writing it after the fact. A log produced after launch is a justification. Write it before, even if it is rough.
  • Listing facts as assumptions. “We use the Kubernetes version we run” is not a belief worth recording. If it cannot be false, leave it out.
  • Never closing items. Every line should end as confirmed, refuted, or consciously accepted. An ever-growing list nobody reads is no better than silence.

One more link between this practice and earlier AIOps writing: if you are weighing whether a predictive approach will hold up, the notes in our discussion of predicting failures in IT systems are a useful companion, and many of the questions there become assumptions to log.

The next time you plan an AIOps change, spend that first hour on the page before touching a dashboard or a model. When the results arrive, good or bad, you will know which beliefs they supported and which they contradicted, and that is the difference between learning from a rollout and simply surviving it.

Related Posts

Reusable AIOps Project Templates: A Beginner Primer

A plain primer on reusable AIOps project templates: what to include, how to adapt one to a new case, and a mini-glossary for newcomers.

Read More

Protect the First Hour of the Day for One AIOps Task: A Step-by-Step Guide

A precise, five-step method for guarding the first hour of an AIOps engineer’s day so one model-tuning task gets done before chat, standup and tickets arrive.

Read More

How One AIOps Team Learned Its Automation in Ten Minutes a Day

A friendly case study of an ops team that stopped guessing why its automation acted and started replaying one decision a day, ten minutes at a time.

Read More

A Shutdown Routine for AI Careers: End the Day, Keep the Spark

A friendly guide for people growing in AI and AIOps roles: four small moves that close the workday cleanly, protect your focus, and quietly speed up your learning.

Read More

Why Postmortems Never Get Written, and the Focus-Block Fix

Incident write-ups stall because the calendar has no long quiet stretch. Here is a diagnosis of the cause and a step-by-step focus-block remedy for ops teams.

Read More

Incident Notes: What AIOps Confirms and What It Only Suggests

Machine-generated root-cause suggestions read like verdicts. Here is how to label each claim known, assumed or guessed before you change production.

Read More
0 0 votes
Article Rating
Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x