Between Sunday, 16 April 04:00 UTC and Monday, 17 April 17:00 UTC, some customers experienced irregular session durations, causing them to get logged out, sometimes repeatedly.
One of our servers got in a state that caused it to give out invalid session cookies. This resulted in customer web sessions to end prematurely which required users to repeatedly log in again. After we identified which server was in this state, we remediated the problem by restarting the server.
During the incident, no other systems were impacted and notifications went out according to our SLA.
What Are We Doing About This?
We have taken specific measures to prevent this from happening again and will flag it earlier should it reoccur. We are also reviewing our deployment strategy for this application to prevent it from causing irregular behavior in individual application servers.
We regret if this affected your team’s ability to receive alerts in a timely manner. As always, if you have any questions or concerns, feel free to contact us at firstname.lastname@example.org.