We have resolved the issue for now by refreshing the service stack and by adding extended instrumentation to preempt similar situations.
Posted Apr 14, 2020 - 20:39 UTC
We are observing normal operations around API Network. To ensure we can preempt future similar situations, we have deployed additional instrumentation.
Posted Apr 14, 2020 - 20:02 UTC
While we investigate the cause of the outage, normal operations are enabled by serving traffic through alternate environments.
Posted Apr 14, 2020 - 18:12 UTC
At 17:20 UTC, we observed elevated error rates on our API Network feature. Publishing and unpublishing collections (or templates) to API Network and parts of public profile management were culminating in errors.
We have temporarily instantiated a parallel stack to service those calls to restore services. Normal operations have been restored. We are working towards identifying the cause and applying patches if necessary to prevent the outage from recurring.