Grafana Cloud Status
Metrics Alert Rule Evaluation Failures in AWS Australia (prod-au-southeast-1)

Started

Resolved

Duration

1 hr 48 min

Resolvedminor impactVendor link

Update timeline

Resolved

Between approximately 08:00 and 09:40 UTC (about 1 hour 40 minutes), some customers in AWS Australia (prod-au-southeast-1) experienced intermittent failures when evaluating alert and recording rules, which may have caused missed or delayed alert notifications and gaps in recorded series. This has been fully resolved and rule evaluation is operating normally.

Investigating

We continue to investigate intermittent failures when evaluating alert and recording rules in AWS Australia (prod-au-southeast-1), first observed at approximately 08:00 UTC. Some customers may see rules fail to evaluate or be delayed, which can result in missed or delayed alert notifications and gaps in recorded series. We are continuing to investigate. Next update by 11:15 UTC.

Investigating

We are investigating an elevated rate of failures when evaluating Metrics alert and recording rules in AWS Australia (prod-au-southeast-1). Some customers may see rules missed or delayed rule evaluations, which can result in missed or delayed alert notifications and gaps in recorded series. Next update by 10:20 UTC.

Metrics Alert Rule Evaluation Failures in AWS Australia (prod-au-southeast-1) — Grafana Cloud Incident Timeline & Status — DevHelm