System is Healthy

History

Last incident was 17h ago.

Healthy
Incident
Outage

Recent Incidents

Issues with Git HTTP(S) access to repositories
InProgress ‐ More details can be found in the incident issue: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2940
30 Oct 10:47
Resolved ‐ More details can be found in the incident issue: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2940
30 Oct 12:16
Degraded git over https performance
InProgress ‐ - We are seeing elevated error rates for git-https on GitLab.com and are investigating.
29 Oct 19:05
InProgress ‐ - We continue to investigate why we're seeing degraded git over https. More details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937
29 Oct 19:22
InProgress ‐ - Investigations are continuing to identify what is causing the errors.
29 Oct 19:38
InProgress ‐ - We believe that there is little current impact to users so will lower our update frequency, but our team continues to investigate the issue. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937
29 Oct 20:24
InProgress ‐ - Note that due to the other incident, we have paused updates on this incident for the moment though details are in the same issue: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937
29 Oct 22:57
Resolved ‐ - Thank you for your patience, https://GitLab.com has now fully recovered and we have not seen any further issues during our last monitoring period.
30 Oct 00:05
Resolved ‐ - Closing this in favor of the latest incident. Please refer to https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937 for the updates.
30 Oct 03:13
Brief downtime and degraded performance from db failover
InProgress ‐ - We apologize for the brief downtime due to a database failover. We are investigating further.
29 Oct 21:38
InProgress ‐ - After a large spike during the failover, we are seeing performance starting to return to normal though we are analyzing further. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937#note_438640798
29 Oct 21:56
InProgress ‐ - We've confirmed that a GCP restart caused the database failover. Users may continue to see some errors as the system recovers. Details in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937#note_438648004
29 Oct 22:13
InProgress ‐ - We continue to see degraded performance across services, but CI jobs are the most severely impacted. The underlying cause is the database failover and an increase in load across the database cluster. We're identifying the best way to mitigate.
29 Oct 22:31
InProgress ‐ - We are continuing to see an increased load on the database cluster. CI jobs are the most severely impacted. We are considering the best option for mitigation.
29 Oct 22:49
InProgress ‐ - We are seeing the system starting to return to a more normal load and continue to look into whether mitigation is required.
29 Oct 23:09
InProgress ‐ - GitLab is continuing to operate normally. All services appears to have recovered except CI now recovering.
29 Oct 23:16
InProgress ‐ - All the services appears to have recovered and we’re monitoring stability at this time. Further updates will be posted in this issue: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937
29 Oct 23:33
Resolved ‐ - Thank you for your patience, GitLab.com has now fully recovered and we have not seen any further issues during our last monitoring period. Full details at https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2937
30 Oct 00:00
SSO warnings
InProgress ‐ - We’ve received reports that on SSH pushes users were getting an unintended message. This has been remediated.
28 Oct 00:35
Resolved ‐ - Closing incident per updates on https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2916
28 Oct 01:45

Service Information