Keeping an "errors and corrections" knowledge base: entries, runbooks and payoff
One entry per error (symptom, cause, fix, prevention), runbooks for recurring incidents and why it pays off.
The same error solved three times by three people (or by me, six months apart) without anyone remembering the fix: that is the hidden cost of operations. My countermeasure is one file: an "errors and corrections" knowledge base, versioned with the code, where every recurring error has its entry.
One entry per error
Short and concrete. Each entry has a stable id (ERR-042) and five fields:
| Field | Content | Generic example |
|---|---|---|
| Symptom / signature | The exact message or behaviour, easy to search | Login failed for user '<token-identified principal>' |
| Context | Where and when it happens | Batch task run from a managed identity |
| Root cause | Specific, not "permissions problem" | The identity was never created as a user in the target database |
| Selected fix | Actionable, verified | CREATE USER ... FROM EXTERNAL PROVIDER then a minimal role |
| Prevention | An explicit rule, including what not to do again | Check the user in the deployment script |
A Markdown table with one row per error is enough. Add the verification evidence (test, log) and, if an approach was rejected, say so: that stops it from being proposed again.
| ID | Signature | Root cause | Fix | Prevention |
|---------|----------------------------|---------------------------------|----------------------------------|------------------------------------|
| ERR-042 | "Login failed ... token" | User missing from the database | Create the user from Entra | Check step in the deployment |
The lifecycle
flowchart LR
A["Incident"] --> B["Search the signature in the base"]
B --> C{"Found?"}
C -->|yes| D["Apply the confirmed fix"]
C -->|no| E["Diagnose and fix"]
E --> F["Write the test"]
F --> G["Add an entry with the next id"]
D --> H["Update the entry if behaviour changed"]
G --> I["Move the prevention into the repo rules"]
Rules I impose on myself:
- Search before fixing: the signature may already be there.
- A fix for something that comes back gets a test that cites the entry id.
- Next free id: check the whole table. A collision (two different errors under the same number) happened to me once, and it is annoying to untangle.
- Prevention rules live where they apply (repo guide, coding conventions), not only in the table.
Runbooks for recurring incidents
When an incident keeps coming back, an entry is no longer enough: write a short runbook chaining diagnosis and action. The format I keep:
- Trigger (which alert, which symptom).
- Read-only checks, with copy-paste queries.
- Decision tree: transient, third-party change, defect.
- Action and verification of its result.
- Who to notify.
The runbook points to the error entries; the entries point back to the runbook. See also the triage method described in another article.
Why it pays off
- Resolution time: a signature search takes a minute, a full diagnosis an hour.
- Less dependence on individuals: knowledge survives holidays and departures.
- Onboarding: the list of real errors is the best documentation of a system.
- AI assistants: a code-generation tool that reads the base before proposing a change will not re-propose an approach already rejected.
- Trends: seeing which families of errors keep coming back points at the real structural work.
Lessons learned
- Too long and nobody rereads it: an entry is a few lines, narratives go elsewhere.
- An entry without a searchable signature is useless; copy the exact message.
- A copy in the team wiki helps non-developers, but two sources drift apart: name the reference (the repo) and mirror from it, not the other way round.
- Update an entry when behaviour changes: a stale entry is worse than none.