Degradation of Websocket Layer

Incident Report for Dyte

Postmortem

Incident Date: March 3, 2026

Duration: 17:29 – 18:22 UTC (53 minutes)

Service Affected: New Meeting Joins & Recording Initializations

Error Message: {DyteClient} Websocket Network Error

Incident Summary

A large portion of the user base was unable to join new meetings due to failed WebSocket connections to socket.dyte.io. While existing calls remained uninterrupted, new session handshakes failed globally.

Root Cause

The issue was triggered by an inadvertent contract change from our third-party feature flag provider. A breaking change in their SDK/Backend returned an incorrect Data Type, violating the service contract within our connection flow. This prevented the WebSocket upgrade process from completing successfully.

Resolution & Impact

  • Remediation: Our engineering team identified the mismatch immediately, manually disabled the problematic flags, and forced them to safe defaults. Simultaneously, the vendor was engaged to revert their breaking release.
  • Scope: Only users on our legacy stack were affected. Customers already migrated to our Cloudflare Stack remained unaffected, as it bypasses the impacted subsystems.

Preventative Actions

  1. Provider Migration: Moving from the third-party provider to Cloudflare’s internal flag service for increased reliability and tighter integration.
  2. Architecture Update: Accelerating the migration of all remaining customers to the Cloudflare Stack to eliminate dependency on the affected legacy subsystems.
  3. Enhanced Validation: Implementing stricter schema validation to ensure unexpected external data types do not break core connection flows.
Posted Mar 04, 2026 - 17:09 IST

Resolved

We are currently investigating this issue.
Posted Mar 03, 2026 - 22:00 IST