Broken Iframe Embeds
Incident Report for Wistia
Postmortem

On Thursday, September 6th, there was a temporary outage for a portion of iframe embeds, between 5PM and 8:30PM EDT. The affected iframes were those that had not been accessed by any viewers within 24hrs of that time period. Any new requests on these un-cached iframes within the window above would result in the player failing to load.

During preparation for future updates to the player, we introduced a code change altering the formation of metadata that is passed to the player when initializing an embedded video. The metadata change was expected and tested manually, however a particular, unanticipated edge case resulted in a bug for uncached iframe embeds.

Any standard/async/popover embeds were not affected by this issue, and iframe embeds that were accessed by anyone within 24hrs of the problem were also unaffected. Videos viewed within the Wistia application are async embeds and therefore were not affected.

This issue was reported to our engineering team at ~6:30 EDT, and the specific cause was pinpointed and a targeted fix was deployed in just under an hour, with the caveat that the old code had 1hr to fall out of cache. By 8:30PM EDT the updated code was live and the new metadata structure was properly handled, allowing all iframe embeds to continue functioning as normal.

In addition to a general fix to the formation of iframe metadata within our code, our engineers are also updating existing, automated tests to account for the edge case that resulted in this bug. We'll also be evaluating additional QA strategies to be more comprehensive in testing for similar problems going forward.

Posted 2 months ago. Sep 11, 2018 - 13:09 EDT

Resolved
This incident has been resolved.
Posted 2 months ago. Sep 06, 2018 - 22:08 EDT
Monitoring
A broader fix has been rolled out. Most/all cases regards to iframe embeds should automatically resolve in the next hour. We'll continue to monitor for issues.

Clearing your browser cache is recommended.
Posted 2 months ago. Sep 06, 2018 - 20:21 EDT
Identified
A fix has been deployed for iframe embeds. Some iframe embeds may remain non-operational due to caching.
Posted 2 months ago. Sep 06, 2018 - 19:26 EDT
Investigating
We are currently investigating this issue.
Posted 2 months ago. Sep 06, 2018 - 18:52 EDT
This incident affected: Embeds.