OpenTok Was Not Posting Archive Callbacks
Incident Report for TokBox
Postmortem

Timeline

On 2018-06-05 at 14:17 PST there was an error in the deployment of new software which omitted a key database update. Callbacks for status changes for archives were not posted from 2018-06-05 14:17 until 2018-06-06 00:50 PST.

Root-Cause Analysis

This problem was caused by human error.

Remediation

We have revisited both the process for the deployment and have trained relevant staff and are improving the automation for this specific operation.

Recommended

No archives were lost because of this issue. However, if you created an archive during this time and did not receive a callback and not retrieve the archive, you may have archives that we created and are available for download.

We only store archives for 72 hours from the date they were created. After 72 hours, the archives are deleted and cannot be recovered.

You can use the REST API to list archives and get the information needed to download them: https://tokbox.com/developer/rest/#listing_archives

You can then make calls to retrieve the archives: https://tokbox.com/developer/rest/#retrieve_archive_info

Posted 5 months ago. Jun 07, 2018 - 10:51 PDT

Resolved
On 2018-06-05 at 14:17 PST there was an error in the deployment of new software which omitted a key database update.
Callbacks for status changes for archives were not posted from 2018-06-05 14:17 until 2018-06-06 00:50 PST.
The issue is now resolved and archive callbacks are working as expected.
Posted 5 months ago. Jun 06, 2018 - 01:00 PDT
This incident affected: Enterprise (Enterprise Archiving) and Standard (Standard Archiving).