How I Run a Blameless Postmortem

A postmortem is not a record of what went wrong. It is a piece of writing whose only job is to make the same failure less likely the next time. If it doesn't change what happens after, it was theatre.

I write a lot of them, because I maintain a distributed database that real systems depend on, and distributed systems fail in ways that are nobody's fault and everybody's problem. I'm also twelve hours ahead of most of the people I work with. By the time anyone reads what I wrote, I'm asleep. That constraint shaped how I run a postmortem more than any framework did. The document has to stand on its own, because I won't be in the room to defend it. Most of the time there is no room.

So the first rule is the one everyone repeats and few people actually practise: it's blameless. Not as a kindness. As an accuracy requirement. The moment a postmortem starts hunting for the person who broke it, people stop telling you what really happened. They round off the timeline. They leave out the part where they were guessing. You end up with a tidy story and a useless one. Systems fail, people make locally reasonable decisions inside them, and a good postmortem assumes both. If your incident has a villain, you haven't finished investigating. When I write up something that involved a change I shipped, I name myself the same way I'd name anyone else: as a participant in a system that let the failure through, not as a culprit.

The second rule is that it has to start with a timeline, and the timeline has to be boring. Times, in one timezone, in order. What was observed, what was believed at the time, what action was taken. The distinction between observed and believed is the whole game. "Replication lag climbed" is an observation. "We thought it was the network" is a belief, and beliefs are where postmortems earn their keep, because the gap between what people believed and what was true is exactly the thing you're trying to shrink. I write the timeline so that someone who wasn't on the call can reconstruct not just the events but the reasoning. That reader is usually a real person on my team, reading at the start of their day while I'm offline.

Contributing factors, not a root cause

I've stopped using the phrase "root cause." It's a comforting fiction. It implies there's one thing, and if you'd just found it sooner, none of this would have happened. Real incidents don't work like that. They're a chain of small, individually survivable things that lined up. The config that was fine until the load changed. The alert that fired but didn't page. The retry that made a slowdown worse instead of better. Say a follower falls behind during a spike. That's not one bug, it's a backpressure assumption, a monitoring gap, and a recovery path that nobody had exercised under that exact shape of load, all at once.

When you write "contributing factors" instead of "root cause," you give yourself permission to list all of them, and you give the next person several places to intervene instead of one. That matters because you rarely get to fix the deepest cause cheaply. You almost always get to fix the second or third contributing factor cheaply, and that's often enough to break the chain. I'd rather ship the boring solution with a benchmark than the clever one without, and the same instinct applies here. Pick the contributing factor you can actually close, close it, and prove it.

Action items that ship

This is where most postmortems quietly die. They end with a list of good intentions, no owner, no date, and everyone nods and goes back to work. A few weeks later the same failure happens and someone finds the old document and feels embarrassed. I've been that someone.

An action item is only real if it has these three things:

• A named owner: one person, not a team. "The platform team will look into it" is how nothing gets done. One name, accountable, free to delegate.
• A change you can verify: a test that now exists, an alert that now fires, a default that's now safer. "Be more careful" is not an action item. It's a wish.
• A place it lives, a tracked issue, in the backlog, with the same weight as feature work. If it's not in the system where work actually happens, it isn't work.

I treat the action items as the only part of the document that's allowed to outlive the week. The narrative is for understanding. The action items are the product. If a postmortem generates three sharp, owned, shippable changes, it worked, even if the prose is plain. If it generates a beautiful narrative and zero merged changes, it failed, no matter how good the writing felt.

There's a discipline that the async, written-first style forces on all of this, and it's a gift in disguise. When you can't gather everyone in a room and talk it out, the document has to carry the full weight. No tone of voice to soften a hard finding. No "you know what I mean." If the writing is vague, the reader twelve hours away can't ask a quick follow-up and wait for the answer; they're blocked for a full day. So I write postmortems the way I write code review comments: specific, kind, and actionable, assuming the reader is smart, busy, and not in the room. Code review is the highest-leverage mentoring there is, and a postmortem is just code review for a failure instead of a diff.

None of this requires a tool or a template. It requires you to believe the document has one job and to refuse to let it do anything else. Not to assign blame. Not to perform thoroughness. Not to make the writer look good. Just to make the next instance of this failure a little less likely, for someone who wasn't there and has only your words to go on.

A postmortem you write and never act on is a more expensive way of forgetting.