Tracks & streams

A MediaStreamTrack is a single live source of media: one camera, one microphone, one screen. A MediaStream is a container that groups tracks so they can be passed around as a unit. The track is what flows over the connection. The stream is bookkeeping.

That distinction is the whole model. Everything else (attaching media to a peer connection, muting, switching cameras, rendering the remote side) is built on it. This guide walks the full path: how tracks and streams relate, how a track becomes RTP on the wire, how to swap a source without renegotiating, and how the receiving peer rebuilds what it gets.

If you have not captured media yet, start with Capturing audio and video. This guide assumes you already hold a MediaStream from getUserMedia.

The container and the source

getUserMedia hands you a MediaStream. Inspect it and you find tracks inside.

const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });

stream.getTracks();        // [MediaStreamTrack(video), MediaStreamTrack(audio)]
stream.getVideoTracks();   // [MediaStreamTrack(video)]
stream.getAudioTracks();   // [MediaStreamTrack(audio)]

The stream itself carries almost no data. It holds references to tracks and fires addtrack / removetrack when its membership changes. A track can belong to several streams at once, and a stream can hold zero tracks. The two objects have independent lifecycles.

A track exposes its own identity and state:

Two of these, enabled and muted, look similar and behave differently. They get their own section below, because confusing them is the most common cause of "the call connected but I see nothing".

A track also carries settings and capabilities, which describe what the underlying device is doing and what it could do:

const [video] = stream.getVideoTracks();

video.getSettings();
// { width: 1280, height: 720, frameRate: 30, facingMode: 'user', deviceId: '…', … }

video.getCapabilities();
// { width: { min: 1, max: 1920 }, height: { … }, frameRate: { … }, facingMode: ['user', 'environment'], … }

getSettings() is the only honest answer to "what resolution am I actually capturing". Constraints are a request; settings are the result. Read them after the track is live, never assume them.

A worked example: ask for 1080p and you may get 720p. The browser picks the closest mode the device supports, and getSettings() reports what it actually chose.

const stream = await navigator.mediaDevices.getUserMedia({
  video: { width: { ideal: 1920 }, height: { ideal: 1080 } },
});
const [video] = stream.getVideoTracks();

video.getSettings().width;   // 1280: the camera had no 1080p mode

If your sender SDP and your bandwidth budget assume 1080p but the device delivered 720p, you have silently mis-sized everything downstream. Read settings, then size your UI and your bitrate to the truth.

applyConstraints() lets you renegotiate the source after the fact without touching the track itself:

await video.applyConstraints({ frameRate: { max: 15 } });

This changes what the existing track produces (useful for dropping frame rate on a congested link), and it does not fire negotiationneeded, because the media section's identity is unchanged.

For the full property and method reference, see MediaStream and MediaStreamTrack.

Attaching media to a connection

A track on its own does nothing over the network. To send it, you attach it to an RTCPeerConnection. Two methods do this, and the difference matters.

addTrack

addTrack(track, ...streams) is the common path. Hand it a track and the streams it belongs to. The connection finds or creates a sender for it and returns the RTCRtpSender.

const pc = new RTCPeerConnection({ iceServers: ICE });
const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });

for (const track of stream.getTracks()) {
  pc.addTrack(track, stream);
}

Passing the stream as the second argument is not cosmetic. The stream's id travels in the SDP. When the remote peer receives the tracks, that grouping tells it which video track and which audio track belong together, so it can render one <video> element with synced sound instead of guessing. Skip the argument and the remote side gets orphaned tracks it must reassociate by hand.

addTrack triggers a negotiationneeded event because the set of media to send has changed. That event drives the offer/answer cycle covered below.

addTransceiver

addTransceiver(trackOrKind, init) creates the transport machinery directly and gives you control before any media exists. You can pass a track, or just a kind string to reserve a slot for media you will supply later.

// Reserve a send-only video slot now, fill it after a device picker resolves
const tx = pc.addTransceiver('video', { direction: 'sendonly' });

// later
await tx.sender.replaceTrack(cameraTrack);

addTransceiver is the right tool when you need a receive-only line (a viewer that never sends), when you want to fix the order of media sections, or when you need a sender ready before the track is. addTrack is sugar over it for the everyday "I have a track, send it" case: internally addTrack reuses a compatible transceiver or creates one.

The ordering point is worth dwelling on. Once an offer is created, the order of media sections is locked for the connection's life: you cannot reorder m= lines later. If your application logic expects video in section 0 and audio in section 1 on both peers, declare the transceivers explicitly and in order before the first offer:

const videoTx = pc.addTransceiver('video', { direction: 'sendrecv' });
const audioTx = pc.addTransceiver('audio', { direction: 'sendrecv' });

// Fill them once the devices resolve, in known slots
await videoTx.sender.replaceTrack(cameraTrack);
await audioTx.sender.replaceTrack(micTrack);

This pattern keeps the two peers' transceiver arrays aligned by index, which matters when you address senders by position rather than by searching for a kind. Relying on addTrack alone leaves the order to whatever sequence your code happened to add tracks in.

Transceivers, senders, receivers

Attaching a track creates three linked objects. Understanding their roles makes the rest of the API legible.

• RTCRtpTransceiver: one bidirectional media slot on the connection. It pairs a sender and a receiver and owns the direction. It maps one-to-one to an m= line in the SDP.
• RTCRtpSender: the outbound half. Holds the track you send and controls encoding parameters (bitrate, resolution scaling). replaceTrack lives here.
• RTCRtpReceiver: the inbound half. Surfaces the track the remote peer sends. You read from it; you do not push to it.

const tx = pc.getTransceivers()[0];

tx.sender;     // RTCRtpSender:    what you send
tx.receiver;   // RTCRtpReceiver:  what you get back
tx.direction;  // negotiated intent: 'sendrecv' | 'sendonly' | 'recvonly' | 'inactive'
tx.mid;        // media id, assigned during negotiation: null until then

pc.getSenders(), pc.getReceivers(), and pc.getTransceivers() list them. A transceiver is permanent for the connection's life: you cannot delete it, only stop it via tx.stop(). Stopping it marks the section closed; the slot is not reused, and a stopped transceiver reports currentDirection of null.

The sender also exposes encoding controls that have nothing to do with the track's source. sender.getParameters() and sender.setParameters() adjust bitrate caps and resolution scaling on the wire, independent of what the camera captures:

const sender = pc.getSenders().find(s => s.track?.kind === 'video');
const params = sender.getParameters();

params.encodings[0].maxBitrate = 500_000;        // cap at 500 kbps
params.encodings[0].scaleResolutionDownBy = 2;   // send at half resolution
await sender.setParameters(params);

These shape the outbound RTP without renegotiation and without changing the track. Use them to back off on a congested link while keeping the capture untouched, so a recovery can ramp back up by raising the cap again.

Direction

Each transceiver declares a direction. It controls whether that media section sends, receives, both, or neither.

Direction is negotiated. Set tx.direction = 'sendonly' and the next offer advertises it; the answer agrees on a current direction, which may be narrower than what each side asked for. A broadcaster sets its transceivers to sendonly; its viewers set theirs to recvonly. A symmetric two-way call uses sendrecv on both ends, the default for addTrack.

Changing direction is a renegotiation. It changes the SDP, so it fires negotiationneeded.

How tracks map to SDP

Each transceiver becomes one media section (one m= line) in the SDP offer and answer. Add a video track and an audio track, and the offer carries two media sections:

m=audio 9 UDP/TLS/RTP/SAVPF 111 …
a=mid:0
a=sendrecv
m=video 9 UDP/TLS/RTP/SAVPF 96 …
a=mid:1
a=sendrecv

The a=mid value is the transceiver's mid. It is the stable handle that ties an SDP section to a transceiver across renegotiations: the order of m= lines never changes once set, even if a track is removed. The a=sendrecv line is the direction. The codec payload numbers (111, 96) come from the codec negotiation, which is its own subject. For the full mechanics of how this SDP gets exchanged, see Signaling and renegotiation.

When a track is removed with removeTrack, its m= line does not disappear: it is marked inactive and stays in place, holding its mid and its position. The SDP only ever grows; sections are recycled by direction change, never deleted. This is why mid is a dependable key: section 1 is section 1 for the life of the connection, whatever happens to the track inside it.

The takeaway: a transceiver, an m= line, and a mid are three views of the same media slot. Once you see them as one thing, the negotiation flow stops being mysterious. When you read or write SDP by hand (as the manual-paste signaling mode in this project does), you are reading the textual form of the transceiver list, in order, with their directions and codecs spelled out.

Switching source without renegotiation

Mid-call, a user taps "switch camera". The naive fix is to remove the old track, add the new one, and renegotiate. That works and it is the wrong choice. It tears down and rebuilds a media section, runs a full offer/answer round trip through your signaling channel, and causes a visible stall.

RTCRtpSender.replaceTrack(newTrack) swaps the source in place. The transceiver, the mid, the SDP, and the codec stay exactly as they were. Only the bytes feeding the encoder change. No negotiationneeded fires. The remote peer notices nothing in its signaling: frames simply keep arriving from the same media section.

async function switchCamera(pc, currentStream, facingMode) {
  // Grab the new camera
  const newStream = await navigator.mediaDevices.getUserMedia({
    video: { facingMode },
  });
  const newTrack = newStream.getVideoTracks()[0];

  // Find the sender already carrying video and swap its source
  const sender = pc.getSenders().find(s => s.track && s.track.kind === 'video');
  await sender.replaceTrack(newTrack);

  // Update the local preview to the new track
  const oldTrack = currentStream.getVideoTracks()[0];
  currentStream.removeTrack(oldTrack);
  currentStream.addTrack(newTrack);

  // Release the old camera hardware
  oldTrack.stop();

  return newStream;
}

The new track must match the kind of the old one: you cannot replaceTrack an audio source onto a video sender. You can pass null to stop sending without removing the transceiver; the section goes silent but stays negotiated, ready for the next track.

The constraint is that replaceTrack only changes the source. It does not change the negotiated parameters. If the new track needs a codec or a resolution the section never agreed to, you are back to renegotiation. For a same-resolution camera flip or a mic change, that never happens, and the swap is effectively free.

Why replace beats remove-then-add

The remote peer keeps the same <video> element bound to the same incoming track. With remove-and-add, the remote ontrack fires for a brand-new track, the old element goes black, and you have to rewire the UI. Reach for replaceTrack whenever the kind stays the same.

Muting: enabled versus stop

There are two ways to silence a track, and they are not interchangeable.

track.enabled = false keeps the track live but blanks its output. A video track sends black frames; an audio track sends silence. The encoder keeps running, RTP keeps flowing, the connection stays warm, and the device stays held. Flip it back to true and real media resumes instantly. This is mute. It is also free and reversible.

// Mute the microphone: still sending, just silence
const [mic] = stream.getAudioTracks();
mic.enabled = false;

// Unmute
mic.enabled = true;

track.stop() ends the track for good. readyState becomes "ended", the hardware is released (the camera light goes off), and there is no resuming. To use that source again you must call getUserMedia and obtain a fresh track.

// Turn the camera fully off and release the device
const [cam] = stream.getVideoTracks();
cam.stop();   // readyState → 'ended', camera light off, irreversible

Choose by intent. A mute button uses enabled. A "turn camera off so the light goes out" toggle uses stop and then getUserMedia to come back. Many apps get privacy complaints because their "camera off" only sets enabled = false, leaving the device held and the light on.

The muted property is something else

track.muted is read-only and unrelated to enabled. The browser sets it. It means the source is not currently producing data: the camera was grabbed by another app, the OS revoked access, or, for a remote track, packets stopped arriving.

track.onmute = () => {
  // Source went quiet. Show a placeholder; do not tear down.
};

track.onunmute = () => {
  // Data is flowing again. Restore the live view.
};

On a received track, muted is your signal for remote starvation. When the far side mutes by setting enabled = false, it sends black frames, which still count as data, so the receiver may not see muted flip. But when packets genuinely stall (network drop, the remote device taken away), the receiver's track goes muted and onmute fires. Treat onmute as "show a spinner or last-frame placeholder", not as "the call died". The track is still live and may recover.

This split has a practical consequence for how you signal mute state across a call. Because enabled = false does not reliably surface to the remote muted property, you cannot use it as a mute indicator for the other peer. If you want the remote UI to show a "muted" badge when a user mutes, send that intent explicitly over your data channel as an application message. Do not infer it from the track. The track-level muted flag is for detecting involuntary starvation, not deliberate mute. Keep the two concerns separate:

• Deliberate mute → set enabled locally, and send an app message so peers can render a badge.
• Involuntary starvation → read the remote track's muted / onmute, and render a "reconnecting" placeholder.

Conflating them produces a UI that flashes "muted" on every packet hiccup, or one that never shows a real mute at all.

Reconstructing remote media

When the remote peer adds a track and negotiation completes, your connection fires track. Its event carries the incoming RTCMediaStreamTrack, the RTCRtpReceiver and transceiver it belongs to, and the streams the remote side grouped it into.

pc.ontrack = (event) => {
  const [stream] = event.streams;   // the grouping from the remote addTrack
  const video = document.querySelector('#remote');

  // Reuse the same MediaStream object across both tracks of a call
  if (video.srcObject !== stream) {
    video.srcObject = stream;
  }
};

Because the remote peer passed its stream to addTrack, both the audio and video tracks arrive tagged with the same stream id, and event.streams[0] is the same object for both. Assigning it once to a <video> element gives you synced audio and video. The ontrack handler fires once per track (twice for an audio-plus-video call), but srcObject only needs setting once because the stream identity is shared.

If the remote side did not group its tracks, event.streams is empty and you must assemble a stream yourself:

pc.ontrack = (event) => {
  let stream = videoEl.srcObject;
  if (!stream) {
    stream = new MediaStream();
    videoEl.srcObject = stream;
  }
  stream.addTrack(event.track);
};

This is exactly why passing the stream to addTrack on the sending side is worth the keystrokes: it saves the receiver from reassembling the grouping.

Watch the received track's state too. It is a real MediaStreamTrack, so onmute, onunmute, and onended all fire on it. Wire onmute/onunmute to swap a placeholder in and out, as shown above.

The negotiationneeded event

Whenever the set of media to send changes in a way the remote peer must hear about, the connection fires negotiationneeded. That is your cue to run an offer/answer cycle. It fires on addTrack, on addTransceiver, on a direction change, and on removeTrack, and it does not fire on replaceTrack, enabled, or stop, because none of those change the negotiated shape of the connection.

pc.onnegotiationneeded = async () => {
  const offer = await pc.createOffer();
  await pc.setLocalDescription(offer);
  // Send pc.localDescription over your signaling channel
  signaling.send({ type: 'offer', sdp: pc.localDescription });
};

The event is the seam between this guide and signaling. Track changes produce SDP that has to reach the other peer somehow. In this project that "somehow" is one of the SDP-only signaling modes (manual paste, URL hash, or ntfy.sh pub/sub), and the data itself never touches a server. Only the offer and answer travel; once the connection is up, frames go straight peer to peer over the negotiated media sections. The full handshake, glare handling, and renegotiation flow are covered in Signaling and renegotiation.

One caution: negotiationneeded can fire more than once if you change several things in quick succession, and firing it during an in-flight negotiation causes glare. Guard the handler against re-entrancy, or batch your track changes and let it settle. The signaling guide covers the polite-peer pattern that resolves glare cleanly.

Cloning tracks

track.clone() returns an independent track sharing the same source. The clone has its own id, its own enabled state, and its own lifecycle. Stopping the clone does not stop the original, and vice versa.

const [cam] = stream.getVideoTracks();
const preview = cam.clone();

preview.enabled = true;     // independent of cam.enabled
videoEl.srcObject = new MediaStream([preview]);

Clone when you need the same camera rendered locally at full quality while sending a separately controllable copy to peers, for example a local preview that stays on while you mute the sent track. Both draw from one device, so you pay the capture cost once, but you get two switches to flip. Stopping every clone of a source plus the original is what finally releases the hardware.

A concrete case: a "camera off to peers, but keep my own preview" toggle. Send the clone, keep the original local.

const [cam] = stream.getVideoTracks();
const sent = cam.clone();

selfPreview.srcObject = stream;        // original, always live for you
pc.addTrack(sent, new MediaStream([sent]));

// Hide your video from peers without killing your own preview
function hideFromPeers() {
  sent.enabled = false;                // peers get black frames; your preview is untouched
}

The original and the clone share a source but not a state. Disabling one leaves the other producing real frames. That independence is the entire reason to clone rather than reuse one track in two places.

Complete example: publish, switch, mute, receive

The pieces assembled. This publishes camera and microphone, exposes a camera switch and a mic mute, and renders whatever the remote peer sends.

import { setupPeer, waitIce } from '/src/salon/peer.js';

const pc = setupPeer({ onStateChange: (s) => console.log('pc', s) });
let localStream;

// 1. Publish camera + mic
async function publish() {
  localStream = await navigator.mediaDevices.getUserMedia({
    video: { width: { ideal: 1280 }, facingMode: 'user' },
    audio: { echoCancellation: true },
  });
  document.querySelector('#self').srcObject = localStream;

  for (const track of localStream.getTracks()) {
    pc.addTrack(track, localStream);   // pass the stream so the peer can regroup
  }
}

// 2. Switch camera in place, no renegotiation
async function switchCamera(facingMode) {
  const next = await navigator.mediaDevices.getUserMedia({ video: { facingMode } });
  const newTrack = next.getVideoTracks()[0];
  const sender = pc.getSenders().find(s => s.track && s.track.kind === 'video');

  await sender.replaceTrack(newTrack);

  const oldTrack = localStream.getVideoTracks()[0];
  localStream.removeTrack(oldTrack);
  localStream.addTrack(newTrack);
  oldTrack.stop();                     // release the old lens
}

// 3. Mute / unmute the mic: keeps sending silence, instant toggle
function toggleMic() {
  const [mic] = localStream.getAudioTracks();
  mic.enabled = !mic.enabled;
  return mic.enabled;
}

// 4. Render the remote side
pc.ontrack = (event) => {
  const remote = document.querySelector('#remote');
  const [stream] = event.streams;
  if (stream && remote.srcObject !== stream) {
    remote.srcObject = stream;
  }
  event.track.onmute = () => remote.classList.add('starved');
  event.track.onunmute = () => remote.classList.remove('starved');
};

// 5. Renegotiate whenever the track set changes
pc.onnegotiationneeded = async () => {
  await pc.setLocalDescription(await pc.createOffer());
  await waitIce(pc);
  signaling.send({ type: 'offer', sdp: pc.localDescription });
};

Recap

• A MediaStream groups tracks; a MediaStreamTrack is the live source that actually flows.
• addTrack sends a track you already have; addTransceiver reserves and configures a media slot, including receive-only lines.
• A transceiver, its mid, and one SDP m= line are the same media slot seen three ways. The sender is outbound, the receiver is inbound.
• Direction (sendrecv / sendonly / recvonly / inactive) decides which way media flows and is negotiated.
• replaceTrack swaps a source with no renegotiation, no new SDP, no ontrack re-fire. Always prefer it over remove-then-add for same-kind swaps.
• enabled = false mutes a live track (sends silence/black, device held). stop() ends it (device released, irreversible). muted is browser-set and signals source starvation.
• ontrack fires per track; rely on the grouped event.streams[0] to rebuild synced remote media.
• negotiationneeded is the bridge to signaling: it fires when the track set changes, not on replaceTrack or enabled.

Going further

• Capturing audio and video: getting tracks from devices, constraints, device selection.
• Audio processing: gain, mixing, and per-track audio handling.
• Group calls: managing many senders and receivers across multiple peers.
• Signaling and renegotiation: how the SDP from negotiationneeded reaches the other peer, and how to avoid glare.
• MediaStream and MediaStreamTrack: full property and method reference.

Troubleshooting

Remote shows a black screen. Check the received track's muted property. If it is true, the remote source stopped producing data: another app grabbed the device, permissions were revoked, or packets stalled. Wire onmute/onunmute to show a placeholder rather than tearing the call down; the track may recover.

Camera light stays on after "turning off" the camera. You set enabled = false, which keeps the device held. To release hardware you must call track.stop(), and re-acquire with getUserMedia to come back.

Audio and video are not synced on the remote end. The sender did not pass its stream to addTrack. Without the shared stream id, the receiver gets two ungrouped tracks. Pass the stream: pc.addTrack(track, stream).

Switching camera causes a visible stall. You are removing and re-adding the track, forcing a full renegotiation. Use sender.replaceTrack(newTrack) instead: same kind, no SDP exchange, one frame of latency.

negotiationneeded fires repeatedly or the connection deadlocks. Several track changes fired in sequence, or an offer was created mid-negotiation (glare). Guard the handler against re-entrancy and batch changes; apply the polite-peer pattern from Signaling and renegotiation.

replaceTrack rejects. The new track's kind does not match the sender, or the new source needs parameters the section never negotiated. Same-kind, same-resolution swaps always succeed; anything that changes the negotiated shape needs a renegotiation instead.