Capture & recording

The goal

A MediaStream does not have to come from a camera. Any source that produces pixels or samples in the browser can become a track. A canvas animation, a game, a whiteboard, a playing <video>, a decoded <audio> clip: each one synthesizes media that you can route into a peer connection or write to disk.

This page covers two jobs that share the same plumbing:

• Capture: turn a non-hardware source into a MediaStream you can send over an RTCPeerConnection.
• Record: take any MediaStream (synthesized, captured from hardware, or received from a peer) and serialize it into a file the user can download.

Both jobs lean on a small set of APIs. None of them touch the network on their own. Capture produces tracks; recording consumes them. The peer connection in the middle stays the same machinery used for camera calls.

The same stream feeds both consumers at once. You can send a canvas to a peer and record it locally in the same session, because a MediaStreamTrack can have many sinks.

This page assumes you already know how to open a connection. For the camera-and-microphone path, see /calls/capture. For how the resulting tracks travel, see /streaming/encoding and /streaming/topologies.

Capturing a canvas

HTMLCanvasElement.captureStream(frameRate) returns a MediaStream whose single video track is fed by the canvas backing store. Every time the canvas content changes and the browser samples it, a new frame enters the track.

const canvas = document.querySelector('#stage');
const stream = canvas.captureStream(30); // up to 30 fps
const [track] = stream.getVideoTracks();

The argument is a frame-rate ceiling, not a guarantee. The browser samples the canvas at most frameRate times per second. If you draw less often, fewer frames are produced. If you draw more often, frames are dropped to stay under the cap.

The frame-request model

There are two ways the track pulls frames from the canvas. They are mutually exclusive, decided by the argument you pass.

Automatic sampling: pass a positive number. The browser samples the canvas on its own schedule, up to that rate, whenever the canvas is "dirty" (something was drawn since the last sample).

const stream = canvas.captureStream(30);

This fits anything that animates continuously: a game loop, a particle system, a live visualization. You draw on requestAnimationFrame; the track samples in the background.

Manual sampling: pass 0 (or omit a meaningful rate) and drive frames yourself with requestFrame().

const stream = canvas.captureStream(0);
const [track] = stream.getVideoTracks();

function pushFrame() {
  draw();             // paint the current frame
  track.requestFrame(); // emit exactly one frame to the track
}

captureStream(0) produces no frames until you ask. Each requestFrame() call emits the current canvas content as one frame. This is the right model for content that changes only on an event: a whiteboard that updates on pointer moves, a slideshow that advances on a click, a turn-based board game. You avoid sending dozens of identical frames per second when nothing moved.

A continuous animation example

Here is a self-contained loop that animates a canvas and exposes it as a stream. The draw loop runs on requestAnimationFrame; the capture rate is capped at 30 fps.

const canvas = document.querySelector('#stage');
const ctx = canvas.getContext('2d');
const stream = canvas.captureStream(30);

let x = 0;
function frame() {
  ctx.fillStyle = '#000';
  ctx.fillRect(0, 0, canvas.width, canvas.height);
  ctx.fillStyle = '#fff';
  ctx.fillRect(x, 60, 40, 40);
  x = (x + 4) % canvas.width;
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);

// `stream` now carries the animation as a video track.

The draw rate and the capture rate are independent. You can draw at 60 fps and capture at 30; the track samples every other frame. You can draw at 15 fps and capture at 30; the track produces 15 distinct frames and the connection layer holds the last one between draws.

Resolution and resizing

The track's frame size follows the canvas backing store: canvas.width and canvas.height, not the CSS display size. A canvas styled to 1280×720 on screen but sized width="640" height="360" produces 360p frames.

Set the backing store to the resolution you want to transmit:

canvas.width = 1280;
canvas.height = 720;

Resizing the canvas after captureStream() changes the track's dimensions on the fly. The receiving side and any recorder adapt, but mid-stream resolution changes force the encoder to reset, which costs a brief quality dip. Pick a size up front when you can.

Capture rate vs draw rate vs send rate

Three rates sit in a row, and confusing them produces stutter that is hard to diagnose.

• Draw rate: how often your code paints the canvas. You control it, usually through requestAnimationFrame, which targets the display refresh (typically 60 Hz).
• Capture rate: how often the track samples the canvas. Capped by the captureStream(fps) argument.
• Send rate: how often the encoder emits frames onto the network. Set by negotiation and adapted down under congestion. Covered in /streaming/encoding.

Each stage can only pass on what the one before it produced. Draw at 15 fps and the track has nothing new to sample 45 times a second, so the captured rate falls to 15 no matter what ceiling you set. Cap capture at 30 while drawing at 60 and half your painted frames never enter the track. Match the capture cap to your real draw rate to avoid wasting paint work or starving the track.

When nothing on the canvas changes, automatic capture still holds the last frame on the track. The encoder may repeat it or, under WebRTC's keyframe logic, drop the send rate to near zero until something moves. That is desirable (a static whiteboard should not spend bandwidth), but it means a stalled draw loop and an idle canvas look identical on the wire. If frames stop arriving, check the draw loop first.

Drawing off the main thread

A canvas you never display can still be captured. Combine an OffscreenCanvas in a worker with a main-thread canvas, or capture an OffscreenCanvas transferred from a visible one. This keeps a heavy draw loop off the thread that handles input and the connection.

// main thread
const offscreen = canvas.transferControlToOffscreen();
worker.postMessage({ canvas: offscreen }, [offscreen]);
// the worker draws; captureStream still runs on the original element

captureStream lives on the HTMLCanvasElement, not the OffscreenCanvas, so capture stays on the main thread while drawing moves to the worker. The track samples the shared backing store either way. For a game already running its simulation in a worker, this avoids a main-thread draw stall dropping captured frames.

Capturing a media element

HTMLMediaElement.captureStream() returns a MediaStream that mirrors whatever a <video> or <audio> element is currently playing. The element keeps playing locally; the returned stream is a second sink on the same decoded output.

const video = document.querySelector('#clip');
await video.play();          // a media element must be playing to emit frames
const stream = video.captureStream();

Use this to re-stream a local file, a blob: URL, or any source the element can decode, onto a peer connection. The element decodes once; the stream carries the decoded frames and samples.

A few behaviors to keep in mind:

• The stream's tracks track the element. Pause the element and the tracks stop producing frames; resume and they continue.
• Seeking the element jumps the stream too. There is no buffering or smoothing on the captured side.
• If the source is cross-origin and not CORS-cleared, the resulting track is marked as tainted and cannot leave the page: the connection silently sends black frames. Serve the media with permissive CORS headers, or host it same-origin.
• captureStream() on a media element is well supported in Chromium. Firefox exposes mozCaptureStream(). Feature-detect both:

function captureFromElement(el) {
  if (el.captureStream) return el.captureStream();
  if (el.mozCaptureStream) return el.mozCaptureStream();
  throw new Error('Media element capture is not supported here.');
}

The returned stream carries both a video and an audio track when the element has both. That makes a media element the simplest way to get synchronized audio-plus-video without combining tracks yourself.

Combining tracks into one stream

A canvas gives you video only. A game's sound effects, a microphone, or a separate <audio> element gives you audio only. To send both over one connection, assemble them into a single MediaStream.

A MediaStream is a thin container, a named bag of tracks. Construct one from an array of tracks pulled from different sources:

const canvasStream = canvas.captureStream(30);
const audioStream = audioEl.captureStream(); // or a mic stream from getUserMedia

const videoTrack = canvasStream.getVideoTracks()[0];
const audioTrack = audioStream.getAudioTracks()[0];

const combined = new MediaStream([videoTrack, audioTrack]);

combined now has one video track from the canvas and one audio track from the element. Add it to the connection the usual way:

for (const track of combined.getTracks()) {
  pc.addTrack(track, combined);
}

Passing combined as the second argument to addTrack tells the remote side these tracks belong together, so the receiver can group them on one MediaStream.

Mixing several audio sources

new MediaStream() groups tracks; it does not blend them. Two audio tracks in one stream stay two separate tracks. WebRTC sends each in its own RTP stream, and most receivers play only the first.

To merge game music, sound effects, and a microphone into a single audible track, route them through the Web Audio API and capture the mixed output:

const audioCtx = new AudioContext();
const dest = audioCtx.createMediaStreamDestination();

// Each source connects into the same destination node.
audioCtx.createMediaElementSource(musicEl).connect(dest);
const micStream = await navigator.mediaDevices.getUserMedia({ audio: true });
audioCtx.createMediaStreamSource(micStream).connect(dest);

const mixedAudioTrack = dest.stream.getAudioTracks()[0];

dest.stream carries one track holding the summed audio. Combine that single track with your canvas video:

const toSend = new MediaStream([
  canvas.captureStream(30).getVideoTracks()[0],
  mixedAudioTrack,
]);

This is the standard way to broadcast a game with commentary: canvas for video, a Web Audio mix for everything you hear.

Keeping the AudioContext alive

An AudioContext created before a user gesture starts in the suspended state. A suspended context produces silence, so the captured track carries nothing audible. Resume it on the first interaction:

button.addEventListener('click', async () => {
  if (audioCtx.state === 'suspended') await audioCtx.resume();
});

Check audioCtx.state if a recipient reports silence. A running context with a connected destination is the working state; suspended and closed both yield quiet.

Track identity and the addTrack stream argument

When you build a combined stream, the tracks keep their own identities: track.id, track.kind, track.label. The MediaStream is only a grouping. The second argument to addTrack(track, stream) is what tells the remote peer which tracks belong together; the receiver reconstructs a stream with the same grouping in its track event.

Add tracks under the same stream object to keep audio and video synchronized on the far side:

const grouped = new MediaStream();
pc.addTrack(videoTrack, grouped);
pc.addTrack(mixedAudioTrack, grouped);

Group them under different streams and the receiver treats them as unrelated, which can break lip-sync on the playback element. One stream per logical source.

Telling the encoder what kind of content this is

Camera footage and synthetic content compress differently. A webcam frame is noisy and tolerant of motion blur. A whiteboard or a code editor is sharp, high-contrast, and full of fine edges that blur badly under aggressive compression.

MediaStreamTrack.contentHint tells the encoder which trade-off to make.

const videoTrack = canvasStream.getVideoTracks()[0];
videoTrack.contentHint = 'detail';

Valid hints by track kind:

Set contentHint before negotiation when you can, and revisit it if the content changes character: for example, switching a screen share from a slide deck (detail) to a video clip (motion). The hint is advisory; the browser is free to ignore it, but Chromium and Firefox both act on it.

Pair the hint with a sensible capture rate. A whiteboard at captureStream(0) with requestFrame() on each stroke, plus contentHint = 'detail', sends crisp text and spends no bandwidth while the canvas is idle. A game at captureStream(60) with contentHint = 'motion' keeps motion fluid.

Sending a canvas to a peer

Putting capture and the connection together. This sketch streams an animated canvas to a connected peer. It assumes the connection and signaling are already wired up. See /calls/capture for the connection setup and the library in src/salon/ for the handshake.

import { setupPeer } from '/src/salon/peer.js';

const canvas = document.querySelector('#stage');
const ctx = canvas.getContext('2d');

// 1. Synthesize the stream.
canvas.width = 1280;
canvas.height = 720;
const stream = canvas.captureStream(30);
const videoTrack = stream.getVideoTracks()[0];
videoTrack.contentHint = 'motion'; // it is an animation

// 2. Drive the animation.
let t = 0;
function frame() {
  ctx.fillStyle = '#000';
  ctx.fillRect(0, 0, canvas.width, canvas.height);
  ctx.fillStyle = '#fff';
  const x = (Math.sin(t / 30) * 0.5 + 0.5) * (canvas.width - 80);
  ctx.fillRect(x, 340, 80, 80);
  t++;
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);

// 3. Attach the track to the connection (host side).
const pc = setupPeer();
pc.addTrack(videoTrack, stream);

// ...run the signaling handshake to exchange SDP...

On the receiving peer, the track surfaces through ontrack:

pc.addEventListener('track', (event) => {
  const [remoteStream] = event.streams;
  const video = document.querySelector('#remote');
  video.srcObject = remoteStream;
});

The receiver does not know or care that the source was a canvas. To it, this is an inbound video track like any other. The pixels arrive, decode, and paint into a <video> element.

This repo runs WebRTC STUN-only and pure peer-to-peer. The signaling channel exchanges SDP and ICE candidates and nothing else: the canvas frames travel directly between the two browsers over the media path, never through a server. Topology choices for more than two peers are covered in /streaming/topologies.

Swapping the source mid-session

A capture track behaves like any other sender track, so you can replace it without renegotiating. Switch from a canvas to a camera, or from one canvas to another, with replaceTrack on the sender:

const sender = pc.getSenders().find((s) => s.track?.kind === 'video');
await sender.replaceTrack(newCanvas.captureStream(30).getVideoTracks()[0]);

replaceTrack runs locally and does not touch SDP, so the switch is immediate and the connection stays up. This is how a broadcast cuts between a game canvas and a webcam intro without a renegotiation pause. The hardware-track version of the same pattern is covered in /calls/capture.

Stopping a capture cleanly

Capture holds resources: a canvas sampling timer, a Web Audio graph, a decoded media element. Stop the tracks when the session ends so the browser releases them:

for (const track of stream.getTracks()) track.stop();

Stopping a track fires its ended event on the remote side, which lets the receiver tear down its view. A track you forget to stop keeps sampling in the background and shows up as a phantom active stream in the connection stats.

Recording a stream

MediaRecorder consumes a MediaStream and emits encoded media as a sequence of Blob chunks. Those chunks, concatenated, form a playable file.

The source can be anything that yields a stream: a canvas capture, a media-element capture, a microphone, or a stream you received from a peer. The recorder does not distinguish them.

Picking a container and codec

You choose the output format with a MIME type string passed to the constructor. Browser support varies, so probe it first with the static MediaRecorder.isTypeSupported().

function pickMimeType() {
  const candidates = [
    'video/mp4;codecs=avc1.42E01E,mp4a.40.2', // H.264 + AAC, Safari-friendly
    'video/webm;codecs=vp9,opus',
    'video/webm;codecs=vp8,opus',
    'video/webm',
  ];
  return candidates.find((type) => MediaRecorder.isTypeSupported(type)) ?? '';
}

const mimeType = pickMimeType();

Notes on the format landscape:

• Chromium and Firefox record WebM reliably. VP8/VP9 video with Opus audio is the safe default.
• Safari records MP4 (H.264/AAC) and does not produce WebM. Probe for MP4 first if you need a file that plays everywhere without re-encoding.
• An empty string lets the browser choose its own default; the chunks still concatenate into a valid file, but you do not control the container.
• Read back recorder.mimeType after construction to learn what was actually selected.

Wiring up the recorder

Construct the recorder, collect chunks on dataavailable, and assemble a Blob on stop.

function recordStream(stream) {
  const mimeType = pickMimeType();
  const recorder = new MediaRecorder(stream, {
    mimeType,
    videoBitsPerSecond: 2_500_000, // ~2.5 Mbps; tune to content and resolution
  });

  const chunks = [];

  recorder.addEventListener('dataavailable', (event) => {
    if (event.data.size > 0) chunks.push(event.data);
  });

  recorder.addEventListener('stop', () => {
    const blob = new Blob(chunks, { type: recorder.mimeType });
    saveBlob(blob, 'recording.webm');
    chunks.length = 0; // release references
  });

  return recorder;
}

Always guard on event.data.size > 0. The recorder can emit empty chunks, and pushing them wastes nothing but adds noise.

Start, timeslice, and stop

start() begins recording. With no argument, the recorder buffers the whole session and fires a single dataavailable when you call stop().

const recorder = recordStream(stream);
recorder.start(); // one chunk delivered at stop()

Pass a timeslice in milliseconds to receive chunks periodically instead of all at once:

recorder.start(1000); // a dataavailable every ~1000 ms

Each dataavailable then carries roughly one second of media. The trade-offs:

To finish, call stop(). It flushes any buffered data through a final dataavailable, then fires stop.

recorder.stop();

pause() and resume() suspend and continue without ending the recording. The recorder's state reads inactive, recording, or paused.

Saving the result

A Blob becomes a download through an object URL and a synthetic anchor click.

function saveBlob(blob, filename) {
  const url = URL.createObjectURL(blob);
  const a = document.createElement('a');
  a.href = url;
  a.download = filename;
  document.body.appendChild(a);
  a.click();
  a.remove();
  URL.revokeObjectURL(url); // free the blob; the download has already started
}

Match the filename extension to the container you recorded. A WebM blob saved as .mp4 confuses some players. Derive it from the MIME type if you support several:

const ext = recorder.mimeType.includes('mp4') ? 'mp4' : 'webm';
saveBlob(blob, `recording.${ext}`);

The same Blob can also be previewed in place before download:

preview.src = URL.createObjectURL(blob);

Handling recorder errors

The recorder fires error if encoding fails: an unsupported track configuration, a source that stops mid-recording, or a platform limit. Handle it so a failed recording does not silently produce a corrupt file:

recorder.addEventListener('error', (event) => {
  console.error('Recording failed:', event.error);
  chunks.length = 0; // discard the partial recording
});

A recording that loses its source track mid-flight is a common cause. If the canvas is removed from the DOM or the stream's tracks are stopped while recording, the recorder ends. Stop the recorder before you tear down the source, not after.

Why chunks must stay in order

The Blob is valid only if the chunks concatenate in arrival order. The first chunk carries container headers; later chunks carry media that references them. Push to the array in the order dataavailable fires and never sort or reorder. A single chunk dropped from the middle of a WebM stream usually makes the whole file unplayable, because the cues no longer line up.

This is why per-chunk upload for long recordings must preserve order too. Number the chunks as they arrive if the transport can reorder them.

Recording a remote stream vs a local one

The recorder treats a received stream exactly like a synthesized one. The stream from ontrack plugs straight into new MediaRecorder(...).

pc.addEventListener('track', (event) => {
  const [remoteStream] = event.streams;
  const recorder = recordStream(remoteStream);
  recorder.start(1000);
  // ...stop on a user action or when the track ends...
});

The differences are operational, not in the API:

• What you capture. Recording a local stream captures the source before transmission: full quality, no network artifacts. Recording a remote stream captures what arrived after encoding, packet loss, and decode. The remote recording reflects the connection's real condition; the local one does not.
• Track lifecycle. A remote track can end abruptly when the peer leaves or the connection drops. Listen for ended on the track and stop the recorder so you flush a complete file:

const [track] = remoteStream.getVideoTracks();
track.addEventListener('ended', () => {
  if (recorder.state !== 'inactive') recorder.stop();
});

• Consent. Recording another person's stream is a privacy matter. Surface a clear indicator and, where it applies to you, get agreement before you start.
• One stream, both ends. Each peer can record its own local stream and the remote one independently. There is no shared recording; recording is a local act on whatever tracks that browser holds.

Memory considerations for long recordings

A MediaRecorder started with no timeslice holds every chunk in memory until stop(). For a long session that grows without bound and can crash the tab.

Strategies, in rough order of how long the recording runs:

• Short clips (seconds to a couple of minutes). Plain start() is fine. Build one Blob at the end.
• Medium sessions (minutes). Use start(timeslice) and keep accumulating into the chunks array. Memory grows linearly but predictably. Assemble the Blob at stop().
• Long sessions (many minutes to hours). Do not keep chunks in JavaScript memory. Flush each chunk out as it arrives. Upload it, or stream it into the File System Access API or an IndexedDB-backed store:

recorder.addEventListener('dataavailable', async (event) => {
  if (event.data.size > 0) {
    await writable.write(event.data); // FileSystemWritableFileStream
  }
});
recorder.start(2000); // flush every 2 s; nothing held in JS memory

Further points:

• A Blob is backed by browser storage, not the JS heap, but the references in your chunks array still pin it. Clear the array (chunks.length = 0) once the final Blob is built.
• Bitrate drives file size directly. videoBitsPerSecond and audioBitsPerSecond let you cap it. A 1080p30 recording at 5 Mbps is ~37 MB per minute; the same at 2 Mbps is ~15 MB.
• Always URL.revokeObjectURL() after a download or preview. Leaked object URLs hold their blobs alive for the page's lifetime.
• The recorder keeps running across tab blur, but throttled timers and reduced canvas draw rates in a backgrounded tab can starve a canvas-sourced recording of fresh frames. Keep the tab foreground for canvas capture, or accept dropped frames.

Recap

• A MediaStream can come from a canvas, a media element, or any combination of tracks, not only hardware.
• canvas.captureStream(fps) samples automatically up to fps; captureStream(0) plus track.requestFrame() emits frames only when you ask. Use manual frames for event-driven content like whiteboards.
• The track's resolution follows the canvas backing store, not its CSS size.
• mediaElement.captureStream() re-streams a playing <video> or <audio>, including synchronized audio. Feature-detect mozCaptureStream. Watch for CORS taint.
• new MediaStream([...tracks]) groups tracks but does not mix them. Blend multiple audio sources through a Web Audio MediaStreamAudioDestinationNode.
• Set contentHint to "detail"/"text" for sharp synthetic content and "motion" for animation.
• MediaRecorder serializes any stream to Blob chunks. Probe formats with isTypeSupported, collect on dataavailable, assemble on stop, save via an object URL.
• start(timeslice) and per-chunk flushing keep long recordings from exhausting memory.
• Recording a remote stream uses the same API as a local one but reflects post-network quality and needs lifecycle and consent handling.

Going further

• /streaming/encoding: how the tracks you capture are encoded and how bitrate is negotiated on the wire.
• /streaming/topologies: sending one captured stream to more than one peer.
• /calls/capture: the hardware path (getUserMedia, device selection, and constraints).
• /connecting/debugging: inspecting tracks, transports, and stats when capture or playback misbehaves.

Troubleshooting