Update timeline
The issue affecting webhook notification delivery has now been resolved. All queued webhook notifications, including the backlog accumulated during the incident, have been successfully processed and delivered. Throughout the recovery period, Engineering investigated intermittent processing behaviour that caused backlog processing to pause and subsequently resume. Recovery actions were completed successfully, processing stability was restored, and queue volumes returned to normal operating levels. Underlying transactions continued to process normally throughout the incident and no data loss has been identified. Recovery validation has been completed and the service is operating as expected.
Webhook processing remains active and delayed notifications continue to be delivered. Overall queue volumes have reduced by approximately 85%, including both the original backlog and newly generated webhook events. Engineering continues to investigate the intermittent behaviour causing backlog processing to pause and subsequently resume. Whilst recovery is progressing and queue volumes continue to reduce, processing rates have varied during the recovery period and investigation remains ongoing to identify the underlying cause and restore consistently stable processing. Underlying transactions continue to process normally and no data loss has currently been identified.
Webhook processing remains active and delayed notifications continue to be delivered. Overall queue volumes have reduced by approximately 70%, including both the original backlog and newly generated webhook events. Engineering continues to investigate the intermittent behaviour causing backlog processing to pause and subsequently resume. Whilst recovery is progressing and queue volumes continue to reduce, processing rates have varied during the recovery period and investigation remains ongoing to identify the underlying cause and restore consistently stable processing. Underlying transactions continue to process normally and no data loss has currently been identified.
Webhook processing has resumed and delayed notifications are being delivered. Engineering is continuing to investigate an intermittent issue that is causing backlog processing to pause and restart, with recovery efforts focused on maintaining stable processing until all queued notifications have been delivered.
The production fix remains deployed and has addressed the original issue that caused webhook notifications to accumulate. However, processing of the retained backlog has stopped again, and delayed notifications are not currently progressing through the queue. Engineering Teams are actively investigating a secondary issue affecting backlog processing and is working to restore delivery while preserving all queued webhook events. Underlying transactions continue to process normally, but clients will continue to experience delayed webhook notifications until backlog processing resumes. No data loss has currently been identified. We will provide a further update once sustained processing and queue reduction have been restored.
The production fix remains deployed and addressed the original issue that caused webhook notifications to accumulate. However, processing of the existing backlog stopped at approximately 14:40 BST, and delayed notifications are not currently progressing through the queue. Engineering is actively investigating the cause and assessing the safest recovery action while preserving all queued webhook events. Current evidence suggests this remains related to processing the unusually high backlog volume rather than a new issue affecting underlying transactions. Transactions continue to process normally, but clients will continue to experience delayed webhook notifications until backlog processing resumes. We will provide a further update once sustained processing and queue reduction have been restored.
The production fix remains in place and has addressed the original issue that caused webhook notifications to accumulate. During recovery, processing of the existing backlog has slowed due to database load, and Engineering is investigating the safest way to resume and accelerate clearance without placing additional pressure on the service. This is not currently considered a recurrence of the original incident. It relates to processing the unusually high volume of notifications already queued. Underlying transactions continue to process normally, but clients may continue to experience delays in receiving webhook notifications while backlog recovery is completed.
We have mitigated the issue affecting webhook notification delivery. A targeted fix has been successfully deployed to production, and webhook notifications are now being processed and delivered. The queue of delayed notifications is reducing, and current recovery indicators show the service is operating as expected. Underlying transactions, funding events and bank transfers continued to process successfully throughout the incident. Current evidence indicates that webhook notifications were delayed and retained for delivery rather than lost, although delivery validation remains ongoing. No client action is required at this time.
We are currently investigating an issue affecting webhook notification delivery for some Currencycloud customers. Customers may experience delayed or missing webhook notifications for transactions, funding events, and bank transfers. We have received reports from multiple customers and are actively working to determine the scope and cause of the issue. Our teams are investigating with the highest priority and will provide updates as they become available. We apologize for any inconvenience and appreciate your patience while we work to resolve this issue.
# Overview Between the afternoon of Thursday 3 September 2026 and the evening of Friday 4 September 2026 \(UK time\), some customers experienced delays receiving webhook notifications from Currencycloud. During this window, a backlog built in the system responsible for delivering these notifications, which meant that notifications were sent later than expected. The underlying issue was identified, and a fix was deployed on Friday 4 September 2026 \(13:29 UTC\), and the resulting backlog was fully cleared by the evening of the same day. # Client Impact A subset of customers who receive webhook notifications experienced delays of up to approximately 24 hours during this window. Once the fix was deployed and the backlog cleared, the large majority of delayed notifications were delivered automatically with no need for any customer action. In a small number of cases, individual notifications did not resend automatically once the backlog cleared. The Engineering Team proactively identified these cases as part of the post-incident review and have been manually redelivering them. If you believe you are still missing a specific notification, please contact your Currencycloud representative or Support and we will investigate and resend it. No action is required from customers because of this incident. The underlying data and payment records referenced by any delayed or resent notifications were not affected. This incident affected only the timing of notification delivery, not the accuracy of the information contained in them. # Root Cause The delay was caused by a performance issue in the database supporting our notification delivery system, which caused it to run under sustained heavy load. This was triggered by an interaction between a recent internal software change and routine infrastructure maintenance activity, which together created conditions that had not previously occurred in combination. As load increased, the system fell behind in processing new notifications, and a backlog built faster than it could be cleared. Once the root cause was identified, we deployed a fix to resolve the underlying performance issue and took manual steps to clear residual load on the affected database so that the backlog could drain at its normal rate. We have since added additional monitoring to detect this specific class of issue earlier and are implementing further safeguards in our notification delivery pipeline to prevent a recurrence. ##### Currencycloud Confidential