A fix has been deployed to resolve the TLS certificate provisioning issue on the Compliance Page. The service is no longer in a degraded state and is operating normally. We are closely monitoring the situation to ensure full stability.
We have identified the root cause of the TLS certificate provisioning issue affecting the Compliance Page. Our team is actively working on a fix and will provide further updates as progress is made.
We are currently investigating an issue affecting the Compliance Page where TLS certificates are not being renewed or provisioned for new instances. This may result in certificate errors or degraded service for users accessing the Compliance Page. Our team is actively working to resolve this. Further updates will be provided as we progress.
The AWS VPC CloudFront origin incident has been fully resolved. We have updated our infrastructure to use VPC origin again, and all services are back to normal operation across all regions. We apologize for the inconvenience caused and thank you for your patience throughout this incident.
The US Console is now back online. All regions are operational following the application of our workaround. We are continuing to monitor the situation closely as AWS works to fully resolve the underlying VPC CloudFront origin issue. We will provide a final update once we are confident the situation is fully stable.
Our workaround is partially in place. The European region is now back online and fully operational. We are actively working to restore the US region as well and will provide an update once it is back. Thank you for your patience.
While AWS is actively working to resolve the underlying VPC CloudFront origin issue, our team is developing a workaround to restore service availability in the meantime. We will share an update as soon as the workaround is in place.
AWS has confirmed that the root cause of this incident originates from an AWS VPC origin issue impacting CloudFront. You can track the AWS service status here: https://status.aws.amazon.com/rss/cloudfront.rss. Our team is closely monitoring the situation and will provide further updates as AWS works toward a resolution.
We have identified the root cause of the Console issue. The incident is caused by an AWS CloudFront downtime affecting our infrastructure. Our team is actively working on a resolution and will continue to provide updates as the situation develops.
Update: We are still investigating the Console issue. Initial findings suggest this may be related to an AWS CloudFront problem. We are continuing to dig deeper to confirm the root cause and will share more details as soon as they are available.
Update: Our Compliance Pages (EU and US) are confirmed operational. The issue is currently isolated to the Console. Our team continues to investigate and will provide further updates shortly.
We are currently experiencing a downtime affecting all services. Our team is actively investigating the issue and will provide updates as soon as more information is available. We apologize for the inconvenience.
The US Compliance Page and Console are fully operational. This incident has been resolved. We apologize for any inconvenience caused and thank you for your patience throughout.
The US Compliance Page and Console have been restored. We are actively monitoring the services to ensure full stability. Thank you for your patience.
We have identified the root cause of the outage affecting the US Compliance Page and Console. A fix is currently being deployed. We will continue to monitor and provide further updates shortly.
We are currently investigating an outage affecting the US Compliance Page and US Console. Both services are currently unreachable. Our team is actively investigating the issue and will provide updates as soon as possible.
The incident has been fully resolved. All systems are operating normally. We apologize for any inconvenience caused and will conduct a post-mortem to prevent recurrence.
The fix has been applied and the service has been restored. We are actively monitoring the situation to ensure full stability and will confirm once we are satisfied everything is operating normally.
We have identified the root cause of the disruption affecting the US Console and US Compliance Page. Our team is actively working on a fix and will provide further updates as progress is made.
We are currently investigating a disruption affecting the US Console and US Compliance Page. Users may be unable to access these services. Our team is actively looking into the root cause and will provide updates as soon as more information is available.
We have successfully upgraded the infrastructure. Response times are now below 250ms and we are monitoring the systems to ensure stability
The infrastructure upgrade is progressing well. We are continuing work to complete the upgrade and restore full performance.
We have identified the root cause of the degraded performance. The issue is due to increased workload beyond current infrastructure capacity. We will be upgrading the infrastructure to support the new workload requirements.
We are currently investigating reports of degraded performance affecting the EU and US Console and Compliance Page. Our team is actively working to identify the root cause and restore normal service levels.
The scheduled maintenance on the Document feature has been completed successfully. All systems are now fully operational. Thank you for your patience during this maintenance window.
Our scheduled maintenance on the Document feature is taking longer than anticipated. We expect to complete the work within the next hour. We appreciate your patience and will provide updates as we progress.
We identified a new background process that was consuming excessive CPU resources. We have rolled back the changes, implemented a fix, and re-enabled the new feature. All systems are operating normally.
We have identified the issue and are currently upgrading the infrastructure to better respond to the traffic. We are monitoring the situation closely.
We've identified a gateway timeout issue affecting the US Console and Compliance Page. Our team is actively working on a resolution.
Fix has been deployed.
The patch has been reviewed and now we'll deploy it.
The bug only affect access without NDA, we are working on a patch.
After login to the compliance page using magic link the NDA signature trigger a blank page.
All pending certificate renewals have been completed. The incident is now closed.
We're deploying a fix to resume ACME certificate renewing.
The newly deployed version have introduced a non backward compatible state change that causes the previous version to segfault during the certificate renewal process.
We rolled back the load balancer to the previous console version in the US region. The console is now operational. We are continuing to investigate the issue.
All requests to the console API returns timeout.
All traffic has recovered. Our deployment process was in a corrupted state and performed the wrong blue/green swap.
Load balancers sent traffic to outdated targets, which resulted in 5xx responses
Our monitoring detected an increase in 5xx errors across all HTTP responses.