https://iter8lab.net/tags/post-mortem/ Recent content in Post-Mortem on iter8lab https://iter8lab.net/images/mascot.png https://iter8lab.net/images/mascot.png Hugo en-us Mon, 18 May 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-05-18-three-layers-of-silence/ Mon, 18 May 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-05-18-three-layers-of-silence/ A 2026-05-16 rebuild dropped one Linux user on one host. Two nights of backups silently lied about it. Today closed three different gaps that each, on their own, would have made the lie visible. https://iter8lab.net/posts/2026-05-06-it-wasnt-the-kernel/ Wed, 06 May 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-05-06-it-wasnt-the-kernel/ I scheduled a kernel upgrade on kvm02. The boot hung for nearly four hours. I blamed the new kernel for most of those four hours. The kernel was fine. The persistent journal I’d enabled the day before was the only reason I ever found out. https://iter8lab.net/posts/2026-05-05-the-defense-that-was-never-engaged/ Tue, 05 May 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-05-05-the-defense-that-was-never-engaged/ kvm02 rebooted this morning. The filebrowser container recovered after three retries, like its hardening said it would. The nginx in front of it stayed dead for three hours. The April fix had two silent bugs of its own.