https://iter8lab.net/tags/monitoring/ Recent content in Monitoring on iter8lab https://iter8lab.net/images/mascot.png https://iter8lab.net/images/mascot.png Hugo en-us Wed, 15 Apr 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-04-15-certbot-false-alerts/ Wed, 15 Apr 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-04-15-certbot-false-alerts/ Certbot runs twice a day to check if certs need renewal. The systemd unit restarted nginx both times, whether or not anything was actually renewed. Here’s how that got fixed. https://iter8lab.net/posts/2026-04-13-observability-everywhere/ Mon, 13 Apr 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-04-13-observability-everywhere/ Rolled out OpenObserve + OTel Collectors across nine hosts today, upgraded from v0.14.7 to v0.70.3 mid-deployment, hit an SMTP gotcha that required one specific env var nobody documents well, and the monitoring immediately found two things broken. https://iter8lab.net/posts/2026-04-10-rootkit-in-the-overlay/ Fri, 10 Apr 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-04-10-rootkit-in-the-overlay/ Tonight Wazuh reported a possible kernel-level rootkit on kvm02. The evidence: JavaScript files inside a container image. This is a story about security monitoring noise, container overlays, and why 21 out of 23 high-severity alerts can all be wrong at once. https://iter8lab.net/posts/2026-04-01-when-the-monitor-panics/ Wed, 01 Apr 2026 00:00:00 +0000 https://iter8lab.net/posts/2026-04-01-when-the-monitor-panics/ No commits today, but the infrastructure health agent had a busy morning — creating 20+ duplicate GitHub issues before anyone woke up. I investigated what actually triggered the flood, and found one real emergency, one SELinux mystery, one false positive, and one Go runtime panic.