Encoding & adaptation

A camera produces raw frames. A network link carries a finite number of bits per second. Encoding is the step that turns one into the other. WebRTC does this for you, continuously, and adjusts as the link changes.

One encode does not fit every link. A 1080p30 stream at 3 Mbps looks sharp on fibre and falls apart on a congested mobile uplink. The same encode that wastes bandwidth on a tiny thumbnail starves a full-screen view. The encoder has to track the link, and it has to know what you are willing to trade when the link cannot keep up.

This page covers the controls. You read and write them through RTCRtpSender. You verify the result through getStats. The browser handles bandwidth estimation on its own. Your job is to set ceilings and preferences, not to drive the bitrate frame by frame.

The shape of the problem

A single track carries video from one peer to another. Between the MediaStreamTrack and the wire sits an encoder. The encoder compresses each frame, then a packetizer splits the compressed output into RTP packets.

The encoder has knobs. Resolution, frame rate, and target bitrate are the three that matter. They interact. Drop the bitrate and the encoder must either lower resolution, lower frame rate, or raise compression (which lowers quality at the same pixel count). Something gives. The question is what.

The link is not constant. Available bandwidth changes as other traffic comes and goes, as a phone moves between cells, as Wi-Fi degrades. An encode that fits the link at one second overshoots it the next. Overshoot means queuing, then loss, then a stall. So the encoder cannot pick a bitrate once. It tracks the link in real time.

WebRTC ships a bandwidth estimator that does this tracking. You do not run it. You constrain it. You set a ceiling with maxBitrate, you state a preference with degradationPreference, you hint at content with contentHint, and you let the estimator find the operating point underneath your constraints.

RTCRtpSender and the encodings array

Every outgoing track has a sender: RTCRtpSender. The sender exposes the encoder parameters through getParameters() and setParameters().

The relevant field is parameters.encodings. It is an array. Each entry describes one encoded version of the track, one spatial layer. A normal track has exactly one entry. A simulcast track has several.

Read, mutate, write. That is the pattern. You must call getParameters() first, mutate the object it returns, then pass that same object back to setParameters(). You cannot construct the parameters object yourself; the browser fills in fields you are not allowed to set.

const sender = pc.getSenders().find((s) => s.track?.kind === 'video');

const params = sender.getParameters();

// encodings always exists once the sender has negotiated.
if (!params.encodings || params.encodings.length === 0) {
  params.encodings = [{}];
}

params.encodings[0].maxBitrate = 800_000;      // bits per second
params.encodings[0].maxFramerate = 30;         // frames per second
params.encodings[0].scaleResolutionDownBy = 1; // 1 = full resolution

await sender.setParameters(params);

setParameters returns a promise. It rejects if you change a field that cannot be changed after negotiation, or if you alter the number or order of encodings. Treat the read-mutate-write as atomic: do not hold a stale params object across an await and write it later, because another change may have landed in between.

The encoding parameters

maxBitrate is the field you reach for most. It caps the layer. The estimator still lowers the bitrate below the cap when the link cannot sustain it, so the cap only bites when the link has room to spare. Use it to stop a thumbnail from consuming a full-screen budget, or to keep a screen share from saturating an uplink.

scaleResolutionDownBy lets you ship a smaller frame than the capture resolution without touching the capture pipeline. A value of 1 means full resolution. A value of 2 halves each dimension, which quarters the pixel count. This is how you make a 720p camera feed a 360p layer.

active: false pauses encoding for a layer. The encoder stops producing output for it, the bitrate it was using returns to the budget, and you can flip it back to true later, all without an SDP renegotiation. This is the cheap way to mute video bandwidth while keeping the track and transceiver in place.

Simulcast

Simulcast means encoding the same camera into several resolutions at once and sending all of them. You configure it when you add the transceiver, through the sendEncodings option.

const transceiver = pc.addTransceiver(videoTrack, {
  direction: 'sendonly',
  sendEncodings: [
    { rid: 'low',  scaleResolutionDownBy: 4, maxBitrate: 150_000 },
    { rid: 'mid',  scaleResolutionDownBy: 2, maxBitrate: 500_000 },
    { rid: 'high', scaleResolutionDownBy: 1, maxBitrate: 2_000_000 },
  ],
});

Each entry is a spatial layer with its own rid. The rid is a short identifier that travels in the RTP header so the receiver can tell the layers apart. You set rid here and only here: it is fixed for the life of the transceiver.

Order matters. List layers low to high. Some encoders treat the first entry as the base layer.

After negotiation you tune the layers through getParameters/setParameters, same as a single encoding, but you can only change mutable fields like active and maxBitrate, not rid and not the count.

const params = sender.getParameters();
// Pause the high layer; keep low and mid.
params.encodings.find((e) => e.rid === 'high').active = false;
await sender.setParameters(params);

The honest caveat: simulcast needs a forwarding unit

Simulcast pays off when something downstream chooses which layer each receiver gets. That something is a Selective Forwarding Unit, an SFU. The sender uploads all layers once; the SFU forwards the layer that suits each viewer's link, and drops the rest.

This project has no SFU. Media is pure peer-to-peer over public STUN, and signaling carries SDP only. See topologies for why a server never sits in the media path here. In a direct P2P connection there is exactly one receiver and no selector. The receiving peer gets every layer you send, then ignores all but one. You pay the encode cost and the upload bandwidth for layers nobody uses.

So in this project, simulcast is the wrong tool. Encode one layer. Set its maxBitrate to match the link and let the estimator adapt. Reach for simulcast only if you later add a forwarding unit, where one upload serves many viewers at different qualities. The API is here so you recognise it; the deployment that rewards it is not.

Single-layer SVC (scalabilityMode: 'L1T3') is a different case. It adds temporal layers inside one stream and can help a single decoder recover from loss without a second upload. That stays useful P2P. Multi-resolution simulcast does not.

Simulcast versus SVC

Both ship multiple qualities, but they differ in where the structure lives.

Simulcast runs several independent encoders, one per resolution. Each produces a complete, self-decodable stream. The cost is the sum of all layers: three full encodes, three uploads. The benefit is that a forwarding unit can forward any single layer without touching the others, and any encoder that supports the codec can produce them.

SVC (Scalable Video Coding) runs one encoder that produces a layered bitstream. Lower layers decode on their own; higher layers refine them and depend on the lower ones. A forwarding unit drops the top layers to serve a weaker link, keeping the base. The cost is lower than simulcast for the same set of qualities, because the layers share prediction. VP9 and AV1 support spatial SVC; VP8 and H.264 (without the SVC extension) do not.

In this project, neither multi-resolution form helps, because there is no unit to do the dropping. The one structure that still earns its keep is temporal-only SVC (L1T3): a single resolution split into frame-rate layers, which improves loss resilience for a single P2P receiver at almost no extra cost.

What the rid does

The rid (RTP stream identifier) is how layers stay distinguishable on the wire. It is a short ASCII label that rides in an RTP header extension. The receiver reads it to route each packet to the right decoder, and a forwarding unit reads it to decide which layers to forward.

Because rid identifies a negotiated stream, it is fixed at addTransceiver time and read-only after. You can flip a layer's active, raise or lower its maxBitrate, change scaleResolutionDownBy, but you cannot rename a layer, add one, remove one, or reorder them without a fresh negotiation. The SDP carries the set of rid values and their constraints; changing the set is a renegotiation, not a setParameters call.

Codecs

The codec decides how bits map to pixels. Browsers negotiate one in the SDP offer/answer; the chosen codec is the highest-priority entry both sides list. You can reorder the preference list with RTCRtpTransceiver.setCodecPreferences, but you cannot force a codec the other peer lacks.

The tradeoff is quality-per-bit against CPU and hardware support. Newer codecs send the same quality in fewer bits, which directly helps a constrained link. They cost more CPU when the device lacks a hardware path, and software encode at high resolution can overheat a phone or pin a laptop fan.

VP8 and H.264 are the safe floor. Every WebRTC endpoint supports VP8. H.264 has the broadest hardware encode/decode coverage, which matters on battery-powered devices. VP9 buys roughly a third less bitrate at the same quality and brings SVC. AV1 is the most efficient and the most demanding; hardware support is still thin, so software encode limits practical resolution and frame rate.

Pick by constraint. CPU- or battery-bound: H.264 or VP8 for the hardware path. Bandwidth-bound with CPU to spare: VP9 or AV1. When unsure, let the browsers negotiate the default: it is VP8 or H.264 and it always works.

// Prefer VP9, fall back to whatever both sides also support.
const { codecs } = RTCRtpReceiver.getCapabilities('video');
const preferred = codecs.sort((a, b) =>
  (b.mimeType === 'video/VP9') - (a.mimeType === 'video/VP9'),
);
transceiver.setCodecPreferences(preferred);

How the codec is actually chosen

setCodecPreferences reorders your offer; it does not decide the outcome. The decision happens in the offer/answer exchange. The offerer lists every codec it can use, in preference order, inside the SDP. The answerer keeps only the codecs it also supports and answers with its own ordering. The first codec both sides agree on wins for that media section.

Two consequences follow. First, you cannot select a codec the peer never offered: if the other side has no AV1 decoder, AV1 never appears in its answer, and no amount of preference on your side changes that. Second, the codec can differ per direction in theory but in practice browsers negotiate one symmetric codec per media section.

A codec entry is more than a name. It carries a payload type number, a clock rate, and format parameters (profile-level-id for H.264, profile-id for VP9). Two endpoints can both list H.264 yet fail to interoperate if their profiles do not overlap. Constrained Baseline is the profile with the widest hardware support; prefer it when targeting phones.

You can confirm the negotiated codec after connection through getStats: the outbound-rtp report references a codec report by codecId, and that report holds the mimeType actually in use. If the codec is not what you set preferences for, the peer did not offer it.

Congestion control: the part you do not write

WebRTC estimates available bandwidth and adapts the send rate to it. You do not implement this. It runs inside the browser and it is the reason a single maxBitrate cap is usually all the tuning a P2P call needs.

The mechanism is Google Congestion Control. GCC watches two signals. The first is one-way delay variation: when packets start arriving later than they were sent, a queue is building somewhere on the path, which means the send rate is approaching the link capacity. The second is packet loss: sustained loss above a threshold means the link is already overrun. Rising delay makes GCC back off early, before loss. Loss makes it back off harder.

The receiver and sender exchange feedback so the estimator has data to work with. Two feedback formats appear in practice:

• REMB (Receiver Estimated Maximum Bitrate): the receiver computes an estimate and reports a single target bitrate back to the sender in an RTCP message. Older, simpler, receiver-side.
• Transport-Wide Congestion Control (TWCC): the receiver reports per-packet arrival times for every packet across the transport, and the sender runs the estimator. Finer-grained, lets the sender reason about delay precisely. This is the current default in modern browsers.

The outcome: the encoder's target bitrate moves up and down on its own. When you set maxBitrate, you set the ceiling the estimate is allowed to reach. When the link degrades, the estimate drops below your ceiling and the encoder follows it down. You never see the wire rate directly unless you ask for it through getStats, covered below.

This is also why fighting the estimator is a mistake. Do not poll getStats and rewrite maxBitrate every frame to chase the bandwidth. The estimator already does that, with more signal than your script has. Set a sensible ceiling once, adjust it only when the use case changes (full screen vs thumbnail), and leave the second-to-second adaptation to GCC.

How frames become bits

A video encoder does not compress each frame in isolation. It exploits the fact that consecutive frames look almost the same. There are two frame kinds that matter here.

A keyframe (an I-frame, or in VP8/VP9 terms a key frame) is self-contained. It can be decoded with no reference to any other frame. It is large, because it encodes the whole picture from scratch.

A delta frame (a P-frame) encodes only the difference from a previous frame. It is small, because most of the picture did not change. A talking head against a still background is almost all delta frames with the occasional keyframe.

This structure explains several behaviours you will see in the stats. Bitrate is bursty: a keyframe spikes the byte count, then delta frames ride low until the next keyframe. After packet loss, the decoder may be unable to apply further deltas, so the receiver asks the sender for a fresh keyframe (a PLI or FIR message). A new connection always starts with a keyframe. So does a resolution change, because the reference frames are now the wrong size.

This is also why motion costs bits. A static scene produces tiny delta frames. A scene where everything moves (a panning camera, a fast game, confetti) produces large delta frames, because little can be predicted from the previous frame. The same maxBitrate buys a sharp static slide or a soft moving game. The content decides, and contentHint tells the encoder which case to optimise for.

Temporal layering (the T in L1T3) builds on this. The encoder arranges delta frames so that some can be dropped without breaking the ones that follow. Drop the top temporal layer and the frame rate halves but the stream still decodes. This is how an encoder sheds frame rate cleanly under maintain-resolution, and how an SVC stream lets a receiver pick a lower frame rate without a separate encode.

Budgeting a bitrate

A maxBitrate is a budget, and budgets relate to pixels and motion. A rough rule: bitrate scales with pixel count times frame rate times a codec efficiency factor. Doubling resolution (four times the pixels) needs far more bits than doubling frame rate, because each extra pixel must be encoded every frame.

Starting points for VP8/H.264, motion content:

VP9 and AV1 reach the same perceived quality at roughly 30% lower numbers each step. Detail/text content at low motion needs less than motion content, because delta frames stay small. These are ceilings to set, not targets to hit: the estimator settles below them on a constrained link. Set the ceiling at or slightly above what a healthy link can carry, then let GCC find the floor.

degradationPreference

When the link cannot carry the current resolution at the current frame rate, the encoder must sacrifice one of them. degradationPreference tells it which. The field lives on the sender parameters, alongside encodings.

const params = sender.getParameters();
params.degradationPreference = 'maintain-framerate';
await sender.setParameters(params);

maintain-framerate keeps motion smooth and lets resolution drop. Choose it when movement matters more than sharpness: a talking head, a moving camera, a game feed. A blurry-but-fluid 15 fps beats a sharp 5 fps slideshow for anything that moves.

maintain-resolution keeps the picture sharp and lets the frame rate fall. Choose it for static, detailed content where reading the pixels is the point: a shared screen of code, a spreadsheet, a presentation. Nobody minds 5 fps on a slide; everybody minds blurry text.

balanced degrades both in steps and is the reasonable default when content is mixed or unknown.

The browser may also apply a default based on contentHint (next), so set this explicitly when you have a strong preference.

track.contentHint

contentHint tells the encoder what kind of content a track carries, so it can bias its internal tradeoffs. It is a property on the MediaStreamTrack, set directly.

videoTrack.contentHint = 'motion'; // camera, gameplay
// or
screenTrack.contentHint = 'detail'; // 'text' is treated similarly

The hint nudges both the encoder's rate-distortion choices and the default degradation behaviour. 'motion' leans toward maintain-framerate; 'detail' and 'text' lean toward maintain-resolution. Set it on the track as soon as you have the track: it costs nothing and it is the clearest signal of intent. For a screen capture, see capture for where the track comes from and how its source constraints interact with the hint.

contentHint and degradationPreference overlap. The hint is a property of the content; the preference is an explicit override. Set the hint always. Set the preference when you want to be sure.

Verifying adaptation with getStats

You cannot see whether adaptation is working by looking at the video element. You read it from getStats. The outbound-rtp report holds what the sender is actually doing: bytes sent, frames encoded, current resolution and frame rate, and the encoder's current target bitrate.

async function readOutbound(sender, prev) {
  const report = await sender.getStats();
  let row;
  report.forEach((stat) => {
    if (stat.type === 'outbound-rtp' && stat.kind === 'video') {
      row = stat;
    }
  });
  if (!row) return prev;

  // Bitrate is a rate: derive it from the delta between two samples.
  let kbps = null;
  if (prev) {
    const dBytes = row.bytesSent - prev.bytesSent;
    const dTime = (row.timestamp - prev.timestamp) / 1000; // seconds
    kbps = Math.round((dBytes * 8) / dTime / 1000);
  }

  console.log({
    kbps,
    width: row.frameWidth,
    height: row.frameHeight,
    fps: row.framesPerSecond,
    qualityLimited: row.qualityLimitationReason, // 'none' | 'cpu' | 'bandwidth' | 'other'
    rid: row.rid, // present per layer when simulcast is on
  });

  return row;
}

bytesSent is cumulative, so bitrate is the delta between two samples divided by the elapsed time. Sample every second or two; a single reading tells you nothing about rate.

qualityLimitationReason is the field that explains the picture. 'bandwidth' means the estimator has lowered quality to fit the link: adaptation is working as designed. 'cpu' means the encoder cannot keep up and is throttling itself; the fix is a cheaper codec or a lower resolution, not a higher maxBitrate. 'none' means nothing is holding the encoder back. qualityLimitationDurations breaks down how long the sender spent in each state.

Watch frameWidth/frameHeight and framesPerSecond change together with qualityLimitationReason and you can see degradationPreference taking effect: under maintain-framerate the resolution drops while fps holds; under maintain-resolution the fps drops while resolution holds.

For the full stats workflow (which reports exist, how to pair sender and receiver views, and how to read the live numbers in the inspector), see debugging.

A worked setup, end to end

The pieces fit together in a fixed order. Set the content hint on the track, add the transceiver with a single encoding (no simulcast in P2P), cap the bitrate and pick a degradation preference after negotiation, then sample the stats to confirm.

// 1. Hint the content before anything else.
const [videoTrack] = stream.getVideoTracks();
videoTrack.contentHint = 'motion'; // a camera feed

// 2. Add the track with one encoding. Single layer: this is pure P2P.
const transceiver = pc.addTransceiver(videoTrack, {
  direction: 'sendonly',
  sendEncodings: [{ maxBitrate: 1_500_000, maxFramerate: 30 }],
});
const sender = transceiver.sender;

// 3. Negotiate (offer/answer/ICE happen here, over the SDP signaling channel).
//    See /connecting/signaling for the exchange itself.

// 4. After negotiation, set the degradation preference.
//    Motion content: keep frame rate, let resolution fall.
const params = sender.getParameters();
params.degradationPreference = 'maintain-framerate';
await sender.setParameters(params);

// 5. Sample the stats every two seconds to verify adaptation.
let prev = null;
setInterval(async () => {
  prev = await readOutbound(sender, prev);
}, 2000);

Now switch the same code to a screen share and only two lines change. Set contentHint = 'detail' on the screen track, and set degradationPreference = 'maintain-resolution' so text stays sharp while the frame rate drops on a tight link. The capture side of that (getDisplayMedia and its constraints) is covered in capture.

To swap quality at runtime (say the remote view goes from thumbnail to full screen), do not renegotiate. Read, change maxBitrate, write.

function setBudget(sender, bps) {
  const params = sender.getParameters();
  params.encodings[0].maxBitrate = bps;
  return sender.setParameters(params);
}

// Thumbnail: cap low so the layer does not hog the uplink.
await setBudget(sender, 200_000);
// Full screen: raise the ceiling; the estimator climbs into it if the link allows.
await setBudget(sender, 2_500_000);

That covers the whole loop for a single sender: hint, encode, cap, prefer, verify. For more than one remote peer, the same per-sender controls apply to each connection independently. See group calls, where one camera feeds several links and each link may want a different cap.

Recap

• WebRTC encodes each track and adapts its bitrate to the link automatically. You constrain that process; you do not drive it.
• Tune the encoder through RTCRtpSender.getParameters()/setParameters() on the encodings array. Read, mutate, write.
• maxBitrate caps a layer. maxFramerate caps fps. scaleResolutionDownBy ships a smaller frame. active pauses a layer without renegotiation.
• Simulcast encodes several resolutions at once via addTransceiver's sendEncodings. It pays off only with a forwarding unit. This project has none, so encode one layer and cap it.
• VP8/H.264 are the universal, hardware-friendly floor. VP9/AV1 send the same quality in fewer bits at higher CPU cost.
• Congestion control (GCC, with REMB or TWCC feedback) runs in the browser and moves the bitrate for you.
• degradationPreference picks what to sacrifice under pressure: maintain-framerate for motion, maintain-resolution for text and detail, balanced for mixed.
• track.contentHint ('motion', 'detail', 'text') biases the encoder and the default degradation.
• Verify everything through getStats outbound-rtp: bitrate from the bytesSent delta, plus resolution, fps, and qualityLimitationReason.

Going further

• Capture: where tracks and their source constraints come from, before encoding.
• Topologies: why this project stays P2P, and what a forwarding unit would change.
• Group calls: multiple peers, where per-link encoding and degradation choices compound.
• Debugging: reading live bitrate, resolution, and limitation reasons from getStats.

Troubleshooting

Bitrate sits well below maxBitrate. Expected when the link cannot sustain more. Check qualityLimitationReason: 'bandwidth' confirms the estimator is the limit, not your cap. Raising maxBitrate will not help.

Video is sharp but stutters, or smooth but soft. That is degradationPreference working. Flip it: maintain-resolution for sharp-but-low-fps, maintain-framerate for smooth-but-soft. Set contentHint to match the content too.

setParameters rejects. You changed an immutable field, the number of encodings, or their order. Always start from a fresh getParameters(), mutate only active/maxBitrate/maxFramerate/scaleResolutionDownBy, and write back the same object.

encodings is empty or undefined. The sender has not negotiated yet. Read parameters after the transceiver is set up, or guard with if (!params.encodings?.length) params.encodings = [{}].

Simulcast layers all arrive but quality does not improve. In pure P2P there is no selector. The receiver gets every layer and uses one. Encode a single layer instead.

qualityLimitationReason is 'cpu'. The encoder is the bottleneck, not the link. Lower the resolution, lower the frame rate, or negotiate a codec with a hardware path (H.264, VP8). A higher maxBitrate makes it worse.

Codec preference is ignored. setCodecPreferences cannot select a codec the other peer did not offer. Confirm both endpoints list it in getCapabilities, and that it survives the offer/answer in the negotiated SDP.