Upgrade & Secure Your Future with DevOps, SRE, DevSecOps, MLOps!
We spend hours scrolling social media and waste money on things we forget, but won’t spend 30 minutes a day earning certifications that can change our lives.
Master in DevOps, SRE, DevSecOps & MLOps by DevOps School!
Learn from Guru Rajesh Kumar and double your salary in just one year.
Most people new to AIOps expect the hard part to be the model. In practice the hard part is usually a small, boring mismatch between what you believe a system does and what it actually does. Rubber-duck debugging is the cheapest known way to surface that mismatch: you explain the failing thing, out loud, to a listener that cannot interrupt, correct you or nod along. Somewhere in the middle of a sentence you hear yourself say something that is not true. This primer covers the habit itself, a worked example from an AIOps setting, and a short glossary so jargon stays out of your way.
What the technique is, and what it is not
The name comes from a desk toy. A developer keeps a plastic duck beside the keyboard and, when stuck, describes the problem to it line by line. The duck is a prop; an empty chair, a notebook or a muted voice memo works equally well. What matters is the constraint: the listener gives no feedback, so every gap in your reasoning has to be filled by you.
This is a solo exercise: no code review, no pairing session, no escalation. You are not asking anyone to find the bug. You are forcing your own mental model to become a sequence of plain statements, and statements can be checked one at a time.
A worked example: the scoring service that went quiet
Imagine a small AIOps service that reads latency and error-rate samples from a set of internal APIs and returns an anomaly score between zero and one. After a routine release, the scores flatten out near 0.1 and nothing alerts, even though a dependency is visibly slow. A tired engineer starts the duck session.
Statement one: “The collector reads samples from the metrics store every minute.” True, the collector log shows a fetch each minute.
Statement two: “It hands those samples to the scoring service, which compares them with the baseline window.” Here the engineer pauses. Which baseline window? The release changed the default from the last seven days to the last thirty. Thirty days now includes an incident week, so slow responses look ordinary. The model is behaving; the baseline quietly absorbed the abnormal period.
Statement three: “So the score is low because abnormal data is now part of normal.” Saying it aloud makes the next action obvious: restore the shorter window, then replay the last two days of samples and confirm the scores climb again.
Nobody reviewed anything and no dashboard was opened. The duck session simply forced the engineer to name the baseline window, a thing they had stopped thinking about because it had never failed before. For a neighbouring topic, how teams think about busy event streams, the article on taming IT event noise using smart tech looks at the monitoring side of the same territory.
How to run a session in ten minutes
- Pick one failing behaviour and state it in a single sentence, including what you expected and what you saw.
- Start at the input and speak in order: where the data comes from, what shape it has, who touches it next.
- Use complete sentences, not shorthand. If you catch yourself saying “and then it just does its thing”, stop; that phrase marks the gap.
- Whenever you assert something, ask whether you have seen it with your own eyes or are assuming it. Mark each assumption on paper.
- Finish by listing the assumptions and choose the cheapest one to test first.
Resist the urge to open a terminal until the narration is done. Typing commands mid-explanation turns reasoning into poking, and poking is what you were already doing before you gave up and reached for the duck.
A mini-glossary for the first week
- Rubber duck: any silent listener, real or imaginary, used to make you narrate a problem in full.
- Anomaly score: a number a model assigns to an observation to say how far it sits from usual behaviour. Higher means stranger.
- Baseline window: the stretch of history treated as “normal” when scoring new data. Changing its length changes every score.
- Threshold: the score above which the system raises something for a human. A good score with a badly placed threshold still stays silent.
- Feature drift: the gradual change in an input’s typical values, which can make an older model misjudge fresh data.
- Training-serving skew: a difference between how inputs were prepared when the model was built and how they are prepared in production.
- Replay: feeding recorded past data back through the pipeline to see how it would have behaved.
Notice how many of these entries are really “assumptions you might not have written down”. That is the point. AIOps failures are rarely mysterious; they are usually an unstated assumption meeting a changed default.
Where the duck falls short
What the duck exposes is a hole in your own reasoning, never a fact you were not given. If the bug lives in a vendor library whose behaviour you have never read about, no amount of narration will produce the missing knowledge; you will simply notice that you cannot finish the sentence, which is itself useful, because it tells you precisely what to go and look up.
It also works poorly when you are rushed and angry. Slow your speech down. A calm, deliberately boring voice tends to expose more than a fast, frustrated one.
A note on talking problems through in other fields
Engineers do not own this habit. The same pattern, saying a plan aloud to something that cannot reply, turns up in cooking, travel planning and paperwork. Take a site built around several parts, such as this directory-style website on gastroenterology topics, which lists hospitals and doctors plus a blog and forum area. Reading its parts aloud in order is a fair way to notice which stop you skipped.
Keep it small, keep it regular
The returns come from repetition, not from a grand ritual. Do one short session whenever a result surprises you, even a good result. Surprise is the signal that your model of the system and the system have drifted apart. Readers curious how the same site frames data-backed operational choices can try the write-up on how AI helps IT teams reach decisions pairs well with this primer.
Next time a service returns something odd, resist the dashboard for ten minutes. Say the whole path out loud, from input to alert, as though describing it to someone who has never seen the system. The sentence that makes you hesitate is almost always the one worth testing first.