APIs Issue
Incident Report for PagerDuty
Postmortem

Summary

On November 22, 2022, from approximately 13:12 UTC to 13:17 UTC, most requests to PagerDuty’s REST API in the US service region returned 502 “Bad Gateway” responses and webhooks from some integrations may have been delayed. The website, mobile application, events API, and notifications were unaffected.

What Happened

During the investigation and remediation of a minor issue with retrieving log entries, a change was made that prevented our REST API from handling requests and caused delays for webhooks from some integrations. Responders quickly identified the misconfiguration and promptly restored functionality. Functionality was fully restored by 13:17 UTC.

What are we doing about this?

We have identified places where certain kinds of configuration changes are confusing and difficult to validate. We are adding documentation around those configuration changes and investigating ways of making them safer to perform. For any questions, comments, or concerns, please contact us at support@pagerduty.com.

Posted Dec 05, 2022 - 20:22 UTC

Resolved
We have identified and resolved an incident where the PagerDuty REST APIs were down for a short period of time in the US service region. The incident is now resolved, and there is no ongoing impact to customers. Please reach out to support@pagerduty.com if you have any concerns.
Posted Nov 22, 2022 - 13:43 UTC
This incident affected: REST API (REST API (US)).