Every AIOps program eventually hits the same wall: the platform is tuned, the dashboards are live, and yet the hardest problems still get solved by whoever happens to remember a similar outage from two jobs ago. That memory is pooled practitioner experience, and most organisations have no system for collecting it. An AIOps knowledge audit forces the question into the open: when your engineers face something unfamiliar, do they have access to the lived experience of people who already went through it, or are they rediscovering everything from scratch?

This checklist walks through the places where collective experience either accumulates or evaporates. Work through it honestly. Score nothing; just mark each item “we do this” or “we don’t.”

Why an AIOps knowledge audit starts with people, not platforms

Vendors sell correlation engines. They cannot sell you the judgment of forty operators who have each watched a specific failure mode play out under real load. That judgment lives in Slack threads, hallway conversations, and the heads of senior staff. The audit below is designed to find out how much of it your organisation actually retains, because models trained on tickets inherit whatever the humans wrote down — and humans write down very little unless the system makes it easy.

If you are still early in adoption, pair this audit with a plan for bringing AIOps into a team without derailing it, because the cultural groundwork matters more than the tool choice.

Checklist part one: capturing what the crowd already knows

• Post-incident writeups exist for every major event. Not a form filled in to close a ticket — an actual narrative another engineer could learn from. If only the incident commander ever reads them, mark it “we don’t.”
• Writeups are searchable from the alert console. The moment of need is 3 a.m. during a page, not a quarterly review. If an on-call engineer cannot surface “has anyone seen this before?” in under a minute, the knowledge may as well not exist.
• Multiple voices contribute to each review. One author produces one perspective. The valuable patterns emerge when the database engineer, the network lead, and the application owner each add what they saw. Pooled accounts catch what a single rapporteur misses.
• Near-misses get recorded, not just outages. The incident that almost happened carries the same lesson at a fraction of the cost. Teams that only document failures throw away most of their experience.
• Tribal knowledge has a home outside chat history. Chat retention policies quietly delete institutional memory. If the answer to “why is that threshold set to that value?” lives only in a 2023 thread, mark it “we don’t.”

Checklist part two: feeding experience back into the machines

• Alert feedback loops are closed. When an operator marks an alert as noise, does the suppression logic actually change? Collective annoyance is a dataset; most teams waste it. The discipline described in this piece on reducing event noise with smarter tooling only works when human feedback flows back in.
• Runbooks are versioned and attributed. Engineers trust a procedure more when they can see who refined it and why. Anonymous runbooks rot; attributed ones get maintained.
• Enrichment draws on past resolutions. A modern pipeline should attach “the last three times this fired, here is what fixed it” to the alert itself. If your incident management workflow with AI assistance does not surface historical resolutions automatically, you are paying for correlation and ignoring recollection.
• New hires are pointed at the archive on day one. Onboarding is where accumulated experience pays compound interest. A new engineer who reads fifty incident narratives in week one inherits years of operational scar tissue.

Checklist part three: looking outside your own walls

Internal experience has a ceiling — your organisation has only seen the failures your organisation has had. Mature teams deliberately tap external practitioner communities: public postmortem collections, vendor-neutral forums, meetup war stories. The pattern is universal: before any high-stakes, unfamiliar undertaking, people who pool the experience of many others who already made the journey consistently avoid traps that official documentation never mentions.

You can watch the same mechanism work far from the data centre. Consider an SRE weighing an international relocation — a decision with more ambiguity than any migration runbook. Resources like DesiNRI, a community hub for Indians navigating moves abroad, exist precisely because immigration paperwork reads clean on paper and gets messy in practice. Members answer each other’s questions in a forum, country guides come with document checklists assembled from real applications, and country-specific points estimators — covering routes like Canadian Express Entry and the Australian skilled independent pathway — let someone gauge where they roughly stand and which factors might strengthen their profile. The site itself is explicit that these are orientation tools: final eligibility and document requirements always rest with the immigration authority of the country concerned. That is the same humility a good AIOps team applies to its own models — the estimate guides the human; it never replaces the authoritative check.

• Your engineers participate in at least one external community. Reading counts; contributing counts double. Teams that only consume internal knowledge converge on internal blind spots.
• External lessons get translated into internal checks. A pattern reported by another company becomes a monitor in your stack within a sprint, or it becomes nothing.
• Someone owns the scanning function. Rotating a “community watch” duty beats hoping osmosis happens.

Scoring the audit without turning it into theatre

Count the “we don’t” marks, then fight the instinct to remediate every gap simultaneously. Pick the two gaps closest to the money: usually searchable writeups and closed alert-feedback loops. Fix those, run the audit again next quarter, and move down the list. The goal is not a perfect score. The goal is an operation where any engineer, facing anything unfamiliar, can draw on the pooled experience of everyone who faced it before — colleagues, predecessors, and the wider community of practitioners who already paid the tuition.

That is what separates an AIOps deployment from an AIOps capability. The platform correlates events. The community of people around it correlates experience. Audit both, because the second one is the harder asset to rebuild once it leaks away.