SLO processing has fully recovered. All services are now functional.

We've identified and isolated the impacted service and are seeing recovery in query execution. SLO processing remains degraded.

We are currently investigating the degradation of various services in our US region.

On October 20, Honeycomb experienced about 3 hours of degraded service in the US region. From 13:30 to 14:40 UTC, querying, the Honeycomb UI, the management API, and triggers experienced elevated latency and error rates, with the highest impact between 13:55 and 14:10 UTC. Some trigger alerts may have been late or missed during that time. SLO processing suffered delays between 13:26 and 14:10 UTC.  From 13:57 to 14:09 UTC, about half of requests to the event ingest API received 5xx errors. Lesser impact occurred between 13:48 to 13:55 \(1% error rate\) and for approximately minute at 14:15 \(3% error rate\). We’ll publish a more in-depth report on Friday, October 23 which will cover detection, remediation, and outstanding action items.