The WebRTC Protocol Stack

WebRTC is not one protocol. It is a stack of protocols, each solving one part of a single problem: move encrypted media and data directly between two browsers, across NATs and firewalls, without a server in the data path.

The layers stack in a fixed order of dependency. ICE finds a network path. DTLS secures that path and agrees on keys. SRTP and SCTP run on top of the secured path to carry media and data. Each layer assumes the one below it already works. A failure low in the stack stops every layer above it from starting. This page describes each layer: what it does, the shape of its packets or handshake, why WebRTC needs it, and how all of them share one port.

The whole stack exists because of one design choice. WebRTC moves data peer-to-peer, so there is no trusted server in the middle to relay, order, or vouch for traffic. Every guarantee that a client-server protocol gets from its server (addressing, reliability, authentication, congestion control), WebRTC rebuilds between two untrusting endpoints on a hostile network. Each protocol in the stack rebuilds one of those guarantees.

The stack at a glance

Every WebRTC packet rides over UDP. On top of UDP, the browser runs a connectivity layer, a security layer, and a transport layer. The transport layer splits into two paths: one for media, one for data.

The two security paths differ, and the difference matters. SCTP data runs inside the DTLS record layer: DTLS encrypts and decrypts every SCTP packet. SRTP media does not ride inside DTLS records. It takes only the keying material DTLS negotiated (a mechanism called DTLS-SRTP) and then encrypts media frames with its own lightweight framing, sending them straight over UDP. Both paths still converge on the same port and the same selected network path.

UDP, and the TCP reality

UDP is the base transport. It delivers datagrams with no connection setup, no ordering, no acknowledgement, and no retransmission. A UDP datagram has an eight-byte header (source port, destination port, length, checksum) and then the payload. That is the entire contract: best-effort delivery of independent datagrams.

That minimalism is the right default for real-time media. The alternative, TCP, retransmits any lost segment and delivers bytes strictly in order. Strict ordering means a single lost packet stalls every packet behind it until the loss is recovered: head-of-line blocking. For a live stream this is the wrong trade. A video frame recovered 400 ms after it was due is useless; the decoder has already advanced, and a late frame cannot be shown without rewinding playback. UDP lets the browser observe the loss, skip it, and keep the stream current. Loss degrades quality; it does not stall the clock.

UDP's other property matters for connectivity: it is connectionless, so a NAT cannot track a "connection" the way it tracks TCP. NATs build UDP mappings from observed outbound traffic and time them out quickly. This is why the connectivity layer above has to actively discover and refresh mappings rather than assume a stable tunnel.

UDP is not guaranteed to reach every network. Some corporate and mobile firewalls block UDP outright, allowing only TCP on a few ports. WebRTC has answers: ICE can gather TCP candidates, and a peer can tunnel the whole stack over TCP, or over TLS on port 443 disguised as ordinary HTTPS, when UDP is blocked. Those TCP and TLS fallbacks almost always run through a relay, because the same networks that block UDP also tend to block direct inbound TCP.

In a STUN-only setup like this project, that relay does not exist. The stack is built for UDP first, and on a network that refuses UDP and direct TCP, the connection does not establish. This is stated plainly rather than hidden: see ICE & NAT for the failure modes. Everything below this section assumes UDP reaches the peer.

STUN: discovering the public address

A browser knows its local IP addresses. It does not know what address the wider internet sees when its packets pass through a NAT. STUN (Session Traversal Utilities for NAT) is the small request/response protocol that answers exactly that question.

The exchange is two messages. The peer sends a STUN Binding request to a public STUN server. The server reads the source IP and port from the packet it received (which is the peer's address after its NAT has rewritten it) and returns that address in a Binding response, in an attribute called XOR-MAPPED-ADDRESS. The address is XORed with a constant so that NATs which naively rewrite address-shaped bytes in payloads leave it alone.

A STUN message is compact: a 20-byte header followed by zero or more attributes. The header carries a message type (Binding request, Binding response), a message length, a fixed magic cookie (0x2112A442, which also seeds the XOR), and a 96-bit transaction ID that matches a response to its request. Attributes are type-length-value triples. That is the whole protocol for address discovery.

// STUN servers are configured as ICE servers. STUN-only means no TURN entries.
const pc = new RTCPeerConnection({
  iceServers: [
    { urls: 'stun:stun.l.google.com:19302' },
    { urls: 'stun:stun.cloudflare.com:3478' },
  ],
});

STUN does a second job inside ICE. Once candidates are exchanged, the connectivity checks that probe each candidate pair are themselves STUN Binding requests, now sent peer-to-peer instead of to a server. These checks add two attributes the server exchange does not: a MESSAGE-INTEGRITY HMAC keyed by the ice-pwd from the SDP, and a USERNAME built from the two peers' ice-ufrag values. That keying is what stops an off-path attacker from forging a check and hijacking the path. The same protocol that discovers an address also validates a path and, later, keeps it alive.

Public STUN servers (Google, Cloudflare) are all this project uses. STUN servers are cheap to run and see only address metadata; no media or data ever passes through them.

ICE: finding and choosing a path

ICE (Interactive Connectivity Establishment) is the framework that decides which network path the two peers actually use. It is not a wire protocol of its own. It is a procedure: gather candidate addresses, exchange them, pair them, test each pair with STUN, then select a pair that works in both directions.

A candidate is one possible address-and-port a peer can be reached at, with a type and a priority. ICE gathers three kinds:

• Host: a local interface address (LAN IP, Wi-Fi, Ethernet, a VPN adapter). Reachable only by a peer on the same network or VPN. Highest priority because it is the most direct.
• Server-reflexive: the public address a NAT assigns, discovered by the STUN exchange above. This is the candidate most cross-network connections use.
• Relay: an address on a TURN relay that forwards traffic. This project does not gather these.

Each candidate gets a priority computed from its type, so ICE prefers direct paths over reflexive ones and reflexive over relayed. Candidates are exchanged through signaling, either bundled into the SDP or sent one at a time as they are discovered (trickle ICE, which lets checks begin before gathering finishes).

ICE then forms candidate pairs (every local candidate against every remote candidate of the same IP family), orders them by combined priority, and runs connectivity checks. A check succeeds only when a STUN request sent over a pair gets a valid response and the peer's own check over that pair also succeeds. Both directions must work, because a NAT can permit outbound while dropping inbound. A pair that passes both ways becomes valid. ICE nominates one valid pair as selected, and all DTLS, SRTP, and SCTP traffic flows over it.

pc.oniceconnectionstatechange = () => {
  // 'checking' → 'connected' means ICE found and selected a working pair.
  // Stuck at 'checking' means no pair passed; 'failed' means all pairs exhausted.
  console.log(pc.iceConnectionState);
};

// Inspect the pair ICE actually chose.
const stats = await pc.getStats();
for (const s of stats.values()) {
  if (s.type === 'candidate-pair' && s.nominated && s.state === 'succeeded') {
    console.log('selected pair', s.localCandidateId, '↔', s.remoteCandidateId);
  }
}

The candidate types, NAT behaviors, and the exact combinations where this fails are covered in ICE & NAT.

Where TURN would sit, and why it is absent

When both peers sit behind NATs that refuse direct connections, no host or server-reflexive pair passes an ICE check. The textbook case is symmetric NAT on both sides: each NAT assigns a different external port per destination, so the address one peer advertises is not the address the other can actually reach. The standard fix is TURN (Traversal Using Relays around NAT): a relay server both peers can reach, which forwards packets between them. ICE gathers relay candidates from the TURN server and selects one when nothing direct works.

This project runs STUN only and ships no TURN relay. The data path must stay strictly peer-to-peer; routing gameplay through a relay would place a server in that path, which the project's constraints forbid. The honest consequence: on network combinations where direct traversal fails, the connection does not establish here, and there is no fallback that papers over it. TURN is out of scope for this stack by design.

DTLS: authenticating and securing the path

Once ICE selects a working path, nothing on it is encrypted or authenticated. Any host on the path could read or forge packets. DTLS (Datagram Transport Layer Security) closes that gap. It is TLS adapted for UDP: the same cipher suites and the same handshake state machine, plus the machinery to survive datagram loss and reordering: explicit sequence numbers on records, retransmission of handshake messages, and fragmentation of large handshake flights across datagrams. It adds that reliability to the handshake only; application data above DTLS stays unordered, so it does not reintroduce head-of-line blocking on media.

DTLS does two jobs in WebRTC, and both are essential.

Authenticate the peer. Each side generates a self-signed certificate when the RTCPeerConnection is created. There is no certificate authority; the trust comes from elsewhere. A fingerprint (a SHA-256 hash of the certificate) is placed in the SDP and carried to the other side during signaling (a=fingerprint:sha-256 ...). During the DTLS handshake each peer presents its real certificate, and each side hashes the certificate it received and compares it to the fingerprint it was promised in the SDP. A match proves the peer that completed the handshake is the same one whose fingerprint arrived through signaling. A mismatch means a different key is in play (a man-in-the-middle) and the connection aborts. This is why signaling integrity is the trust anchor for the whole session, a point developed in security.

Agree on keys. The handshake performs an (EC)DHE key exchange that produces shared secret material neither side could derive alone and no eavesdropper can recover. That material keys two things: the DTLS record layer itself, which encrypts SCTP data, and (via the DTLS-SRTP extension) the SRTP keys for media, exported through the use_srtp extension and a key-derivation step rather than carried as record payload.

The handshake is a sequence of flights over the selected ICE path: ClientHello, then ServerHello with the server's certificate and key share, then the client's certificate and key share and a Finished message, then the server's Finished. WebRTC peers negotiate which side acts as client and which as server through the SDP a=setup: attribute (actpass, active, passive). Lost flights are retransmitted on a timer until both Finished messages verify.

If the DTLS handshake never completes (a dropped flight that keeps timing out, a certificate that does not match its fingerprint, or a path that ICE reported as connected but that silently drops packets), the connection sits in connecting and never reaches connected. A stalled DTLS handshake while ICE shows connected is one of the most common failures; debugging covers how to identify it.

SRTP and SRTCP: encrypted media

Media does not travel as raw frames. The browser splits encoded audio and video into RTP packets, then encrypts each as SRTP (Secure RTP) using keys derived from the DTLS handshake. SRTP keeps the RTP header in the clear (routers and the receiver's demultiplexer need to read it) and encrypts only the payload, then appends an authentication tag so the receiver can detect tampering and replays.

The encryption is a stream cipher (AES in counter mode by default) keyed from the DTLS-derived master key plus a per-stream salt. The cipher's counter is built from the packet's SSRC and sequence number, so each packet gets a unique keystream without sending an explicit nonce. A rollover counter extends the 16-bit sequence number so the keystream stays unique across more than 65,536 packets. The authentication tag (HMAC-SHA1 truncated, or an AEAD tag with the GCM suites) covers the header and encrypted payload; a replayed or altered packet fails the check and is dropped before it reaches the decoder.

SRTCP is the same treatment applied to RTCP, the control and feedback channel that rides alongside media. SRTCP encrypts the report payloads, authenticates them, and adds its own index to defend against replay of control packets.

This project's games run on data channels, not media, so SRTP rarely appears in gameplay. The media demos under /connecting exercise it directly; the game library does not. The next section describes the RTP and RTCP framing that SRTP and SRTCP protect.

RTP and RTCP: framing real-time media

RTP (Real-time Transport Protocol) is the framing that lets a receiver rebuild a continuous stream from datagrams that arrive out of order, late, or not at all. The RTP header is twelve bytes before any optional fields, and each field earns its place:

• Sequence number (16 bits): increments by one per packet. The receiver detects loss from gaps in the sequence and reorders packets that arrive out of order.
• Timestamp (32 bits): the sampling instant of the media in the packet, in units of the codec's clock rate. It drives playback timing, sizes the jitter buffer, and synchronizes audio with video (lip-sync) across two independent streams.
• SSRC (32 bits): synchronization source identifier. Names which stream a packet belongs to when several streams share one port, which is the normal case under BUNDLE.
• Payload type (7 bits): an index into the codecs negotiated in the SDP (Opus, VP8, VP9, H.264, AV1). It tells the receiver how to decode the payload.
• Marker bit (1 bit): codec-specific, often flagging the last packet of a video frame so the decoder knows the frame is complete.

The receiver feeds these packets into a jitter buffer. The buffer holds incoming packets briefly, reorders them by sequence number, and releases them on a schedule paced by the timestamps. It trades a little latency for smooth playback: too small and reordering fails under jitter, too large and the stream lags. The buffer sizes itself dynamically from observed network variation.

RTCP (RTP Control Protocol) is the out-of-band feedback that travels next to the media. Senders emit Sender Reports carrying the mapping between the RTP timestamp clock and wall-clock time, which is what makes cross-stream synchronization possible. Receivers emit Receiver Reports carrying observed packet loss, interarrival jitter, and the data needed to compute round-trip time. WebRTC adds feedback messages on top:

• PLI (Picture Loss Indication) and FIR (Full Intra Request) ask the sender for a fresh keyframe after loss corrupts the decode.
• NACK requests retransmission of specific lost packets, used selectively where a quick resend still beats the playout deadline.
• REMB and transport-wide congestion control feedback report the bandwidth the receiver estimates is available.

A sender uses this stream to adapt continuously: lower the bitrate when loss climbs, raise it when the path is clean, send a keyframe when a receiver reports it lost the reference frame. This control loop is how WebRTC media degrades gracefully instead of stalling, the visible counterpart to SCTP's congestion control on the data side.

The split between RTP and RTCP is deliberate. RTP carries the media on a tight, low-overhead path. RTCP carries the slower control loop on its own cadence. Encrypted, they become SRTP and SRTCP, and under rtcp-mux they share the media's single port rather than the historical second port.

SCTP over DTLS: the data channel transport

RTCDataChannel is carried by SCTP (Stream Control Transmission Protocol), encapsulated inside DTLS. SCTP supplies what UDP lacks and what arbitrary application data needs but real-time media does not.

Message framing. SCTP delivers discrete messages, not a byte stream. Send one JSON object, the peer receives exactly one JSON object: no manual length-prefixing or boundary parsing. Large messages are fragmented into chunks on the wire and reassembled by SCTP transparently, so the application never sees a half-message.

Multiplexing. Many data channels share one SCTP association, each assigned its own stream identifier. A game can run state on one channel and chat on another over the same connection, with no extra handshake and no second port. The streams are independent: backpressure or loss handling on one does not stall another.

Configurable reliability. Each channel chooses its delivery guarantee at creation. This is the central design decision for any data-channel application, because it picks where on the spectrum between TCP's guarantees and UDP's speed the channel sits.

// Reliable + ordered (default): like TCP. Every message arrives, in order.
// Reintroduces head-of-line blocking: fine for chat, costly for live state.
pc.createDataChannel('chat');

// Unreliable, no retransmits: deliver once or drop. Best for state that is
// replaced every frame, where a stale packet is worthless.
pc.createDataChannel('state', { ordered: false, maxRetransmits: 0 });

// Partially reliable by time budget: retransmit for up to 100 ms, then give up.
pc.createDataChannel('cursor', { ordered: false, maxPacketLifeTime: 100 });

// Reliable but unordered: every message arrives, order not guaranteed.
// Good for independent events where arrival order does not matter.
pc.createDataChannel('events', { ordered: false });

Set maxRetransmits or maxPacketLifeTime, never both: they are mutually exclusive partial-reliability policies. A reliable ordered channel makes the same trade TCP does, which is correct for a transcript and wrong for sixty-times-a-second position updates, where a dropped packet should be skipped rather than block fresher data behind it. Picking the right mode per channel is the core trade-off; data channels develops it. The client library in src/salon/ opens these channels, and its protocol.js helper frames typed messages over them.

After negotiation, read back what the channel actually agreed to. The values are settled once the channel opens:

const ch = pc.createDataChannel('state', { ordered: false, maxRetransmits: 0 });
ch.onopen = () => {
  console.log(ch.label, ch.ordered, ch.maxRetransmits, ch.id);
  // ch.id is the SCTP stream identifier assigned to this channel.
};

// Backpressure: stop sending when the SCTP send buffer is filling up.
ch.bufferedAmountLowThreshold = 64 * 1024;
function sendWhenReady(msg) {
  if (ch.bufferedAmount > 1024 * 1024) {
    ch.addEventListener('bufferedamountlow', () => sendWhenReady(msg), { once: true });
    return;
  }
  ch.send(msg);
}

SCTP also brings congestion control. It tracks loss and round-trip time across the association and adjusts its send rate so a fast sender cannot overwhelm a slow path or a slow receiver: the same self-restraint TCP applies, which UDP omits entirely. The bufferedAmount property exposes that flow control to the application: when it grows, the sender is outrunning the link, and well-behaved code waits for bufferedamountlow before sending more. For data that must arrive intact, this control is necessary; for media, SRTP's bitrate adaptation through RTCP does the equivalent job, which is one reason media does not run over SCTP.

SCTP is also the reason a data channel can open without a separate network handshake. The DTLS association is already up; SCTP runs an association handshake inside it (a four-way INIT / INIT-ACK / COOKIE-ECHO / COOKIE-ACK exchange) once, and every subsequent createDataChannel is a lightweight in-band negotiation of a new stream identifier rather than a new connection.

One port for everything: BUNDLE, rtcp-mux, and the 5-tuple

A naive WebRTC implementation would open a separate UDP port for each media track, a second port for each track's RTCP, and another for the data channel. Every port is a separate NAT hole to punch, a separate ICE gathering and checking cycle, and another way for traversal to fail. WebRTC collapses all of it onto one port through two mechanisms.

rtcp-mux sends RTCP on the same port as the RTP media it controls, instead of the historical convention of RTCP on the next port up. The receiver tells RTP and RTCP apart by the packet-type byte in the header. One media stream, one port, both directions of traffic. The SDP signals it with a=rtcp-mux, and rtcpMuxPolicy: 'require' refuses to fall back to a second port.

BUNDLE goes further: it places every media track and the data channel onto a single ICE transport and therefore a single port. Audio, video, and the SCTP data channel all ride the one path ICE selected and share the one DTLS session that secured it. The SDP negotiates this with an a=group:BUNDLE line listing the media-section identifiers and an a=mid: tag on each section; the receiver routes packets to the right track by SSRC and MID rather than by port. Modern WebRTC negotiates max-bundle, putting everything on one transport from the first offer.

The result is a connection that lives on a single 5-tuple: source IP, source port, destination IP, destination port, and the transport protocol (UDP). That 5-tuple is the connection's entire identity to every NAT and firewall on the path. One 5-tuple means exactly one mapping to create and keep alive, exactly one hole to punch. Fewer holes is a higher probability that traversal succeeds, which is the concrete, practical reason BUNDLE and rtcp-mux matter for a peer-to-peer connection rather than a server-mediated one: the server case can afford extra ports because the server has a stable public address, while a peer behind a NAT cannot.

When several protocols share one port, the receiver demultiplexes by inspecting the first byte of each incoming packet. STUN, DTLS, and SRTP/RTP occupy distinct, non-overlapping ranges of that first byte by deliberate design: STUN messages begin with the top two bits clear (values 0 to 3), DTLS records begin in the 20 to 63 range, and RTP/SRTP begin at 128 and above. The browser reads one byte, routes each packet to the correct handler, and never confuses a STUN check for a media frame or a DTLS record for either. Because SCTP rides inside DTLS, it is demultiplexed at a second level, after DTLS decrypts the record.

// max-bundle keeps audio, video, and data on one transport / one port,
// and rtcp-mux keeps media and its control on that same port.
const pc = new RTCPeerConnection({
  iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
  bundlePolicy: 'max-bundle',
  rtcpMuxPolicy: 'require',
});

Recap

• UDP is the carrier: fast, lossy, connectionless, the right default for real-time. TCP/TLS fallback exists but needs relay infrastructure this project lacks.
• STUN discovers the public address a NAT assigns and, peer-to-peer, validates and keeps paths alive.
• ICE gathers candidates, tests pairs in both directions, and selects one working path.
• TURN would relay traffic when no direct path exists, but this STUN-only project omits it, so some network combinations simply cannot connect.
• DTLS authenticates peers by matching a certificate fingerprint carried in the SDP, then derives the encryption keys for everything above it.
• SRTP/SRTCP encrypt media and its feedback; RTP/RTCP supply sequencing, timing, stream identity, and the loss/bitrate control loop that lets media degrade instead of stalling.
• SCTP over DTLS carries data channels with message framing, multiplexing, per-channel reliability, congestion control, and bufferedAmount backpressure.
• BUNDLE and rtcp-mux put everything on one port and one 5-tuple, which is what makes NAT traversal tractable for a peer-to-peer link.

Going further

• Architecture: how these layers map onto RTCPeerConnection and the JavaScript API surface.
• Signaling: how SDP carries codecs, ICE credentials, and the DTLS fingerprint between peers.
• ICE & NAT: candidate types, NAT behavior, and exactly where STUN-only traversal fails.
• Security: why the DTLS fingerprint makes signaling integrity the trust anchor for the session.
• State machine: the connection states each layer drives and the order they transition in.
• Data channels: choosing reliability modes and framing messages over SCTP.

Troubleshooting

• Stuck in connecting, ICE reached connected. The DTLS handshake is failing. Check that the certificate fingerprint matches between the SDP each side received and the certificate presented; verify a handshake flight is not being dropped by the path ICE selected.
• ICE never leaves checking. No candidate pair passed a bidirectional connectivity check. Likely a NAT combination that needs a relay this project does not provide.
• Data channel drops messages under load. The channel is reliable and ordered, and head-of-line blocking is stalling it. Move latency-sensitive traffic to maxRetransmits: 0 or a maxPacketLifeTime budget.
• Send throughput collapses, bufferedAmount climbs. The sender is outrunning SCTP's congestion control. Gate sends on bufferedamountlow instead of sending in a tight loop.
• Connection works on LAN, fails across networks. Host candidates pair on the LAN; across NATs you need server-reflexive pairs, and if both NATs are symmetric no direct pair exists.
• Audio plays but video freezes after a glitch. A keyframe was lost and the decoder has no reference. The receiver's RTCP PLI/FIR should request one; persistent freezes point to feedback not reaching the sender.
• One data channel stalls but others keep flowing. Expected. SCTP streams are independent; backpressure on one channel does not block another on the same association.