Elastic Cloud (Public) Status
Replication bug causing slow recoveries

Started

Ongoing

Duration

21 days

Monitoringmajor impactVendor link

Update timeline

Monitoring

Elasticsearch 9.5.3 has been released and all customers currently running 9.5.0, 9.5.1 or 9.5.2 are advised to update to this new version as soon as possible. Impacted customers are advised to follow the workaround steps from https://github.com/elastic/elasticsearch/issues/158212 after the upgrade.

Monitoring

Elasticsearch 9.5.3 has been released and all customers currently running 9.5.0, 9.5.1 or 9.5.2 are advised to update to this new version as soon as possible.

Identified

The fix has been identified and backported to the Elasticsearch 9.5 patch branch. The teams are performing the needed validations before scheduling and rolling out the 9.5.3 release.

Identified

We have identified the root cause and the team is working on a fix scheduled to be released in Elasticsearch version 9.5.3 What you can do: - If you have not yet upgraded to 9.5.*, you are not affected as upgrades to 9.5 have been disabled until 9.5.3 is released - If you are already running 9.5 and experience the symptoms of this issue, please contact Elastic Support. More information can be found on https://www.elastic.co/docs/release-notes/elasticsearch/known-issues#elasticsearch-9.5.2-known-issues

Investigating

We are aware of a bug affecting customers on Elasticsearch versions 9.5.0, 9.5.1 and 9.5.2 where bulk index operations processed on the primary shard can result in divergence on the replicas resulting in high disk usage and long-running recoveries. The team is working on a fix. More information can be found on https://www.elastic.co/docs/release-notes/elasticsearch/known-issues#elasticsearch-9.5.2-known-issues The next update will be in the next 2 hours or sooner if needed.

Replication bug causing slow recoveries — Elastic Cloud (Public) Incident Timeline & Status — DevHelm