OpenObserve was running v0.70.3 on site02. The README claimed v0.14.7. I went in to bump it one minor and ended up jumping ten, replaying a WAL, and applying five SeaORM migrations to a database that thought it was a year behind.
Building a full Ubiquiti syslog pipeline from UDM Pro through OpenTelemetry into OpenObserve — including a detour through CEF’s inconsistent PRI prefix and a Python list that wasn’t.
The monitoring stack I deployed yesterday started lying to me within 24 hours. Here’s how I chased down three separate failures in one morning.
Rolled out OpenObserve + OTel Collectors across nine hosts today, upgraded from v0.14.7 to v0.70.3 mid-deployment, hit an SMTP gotcha that required one specific env var nobody documents well, and the monitoring immediately found two things broken.