2020.05.11 - Issues with subscriptions and payments

Summary

A change in the Auth0 Management Dashboard triggered errors when users attempted to load the subscription and payment related Dashboard tabs.

What Happened

A change to the Auth0 Management Dashboard was deployed to support an upcoming feature. This change inadvertently introduced an unsupported attribute to select Auth0 Management API requests, triggering validation error responses as expected. The failed requests blocked the processing of a narrow subset of Dashboard functions, specifically the loading of subscription/payment information when accessing the related tabs. All Auth0 APIs and services continued to function throughout the incident.

Mitigation Actions

We corrected the functionality that triggered the malformed request.
We are increasing monitoring for these Dashboard features.

Events Timeline

17:30 UTC - Issue loading Subscription and Payment tabs reported.
17:37 UTC - Confirmed the underlying errors triggering the failures.
17:41 UTC - Posted to status page as investigating.
17:46 UTC - Initiated a rollback of the recent Dashboard deployment.
17:51 UTC - Identified the functional change that triggered the errors and initiated steps for a permanent resolution.
17:56 UTC - Status page updated to Monitoring.
17:58 UTC - Confirmed billing functions are working.
18:26 UTC - Incident marked as resolved.

Posted May 27, 2020 - 19:22 UTC

Resolved

This incident has been resolved.

Posted May 11, 2020 - 18:26 UTC

Monitoring

A fix has been implemented and we are monitoring the results.

Posted May 11, 2020 - 17:56 UTC

Investigating

We are investigating an issue for self-service customers who wish to change their subscription level, change or view their payment methods, and download invoices from the Dashboard. The relevant Dashboard tabs for these operations are currently unavailable. Authentication operations, as well as tenant management operations are not affected by this. We will keep you updated as we progress.

Posted May 11, 2020 - 17:41 UTC

This incident affected: Management Dashboard (manage.auth0.com).