O365 log ingestion issue impacting some accounts with delay and log degradation alerts.
Update timeline
Resolved
This incident has been resolved.
Monitoring
Backlogs have been processed and we are verifying recovery is complete
Identified
Microsoft had a service issue on Sept-13 under advisory [MO1471241] which caused a backlog of logs. After restoration of MS service, the age and volumes of logs caused complications for XDR backend processing. Processing is now recovering. Affected customers may receive alerts based on older data.
Investigating
We are experiencing an O365 log ingestion issue impacting some proportion of accounts. The affected customers have not receiving logs for the past 3 days and are seeing log degradation tickets. SRE and Data Engineering teams are actively troubleshooting the issue.