Camera and screen together

A presenter wants to show slides and their face. The slides come from getDisplayMedia. The face comes from getUserMedia. Both are video tracks. The question is how to get both across one peer-to-peer connection so the viewer sees the screen large and the presenter in a corner.

There is no single API for "send a screen and a camera". There are three strategies, and they differ in what reaches the remote peer, what they cost, and whether they renegotiate. Picking the wrong one shows up as a stall, a missing video, or a melted CPU.

This guide covers all three end to end:

1. Swap: one outgoing video track, toggled between camera and screen with replaceTrack. Instant, no renegotiation, but only one source is visible at a time.
2. Two tracks: camera and screen as two separate video transceivers. The viewer gets two streams. Adding the second one renegotiates, and the receiver must work out which is which.
3. Composite: draw both into a canvas, picture-in-picture, and send the canvas as one track. Full layout control, one transceiver, but you pay CPU every frame.

If you have not captured either source yet, start with Display capture for the screen and Capturing audio and video for the camera. This guide assumes you already hold both streams. It builds directly on the track model in Tracks and streams. Read that first if replaceTrack, transceivers, and mid are not yet familiar.

The goal, stated precisely

"Show the screen and the presenter" hides three different targets. Name yours before writing code.

• One at a time. The viewer sees either the screen or the face, and the presenter toggles. A talk that mostly shows slides but cuts to the speaker for questions. Strategy 1.
• Both, laid out by the viewer. The viewer's client decides where the face sits relative to the screen: maybe a sidebar, maybe a corner it can drag. Two independent videos. Strategy 2.
• Both, laid out by the presenter. The presenter composes the picture-in-picture and the viewer receives it as a finished frame. The layout is fixed at the source. Strategy 3.

Each maps to one strategy. The rest of the guide is the mechanics of each, the cross-link between renegotiation and "adding a track", and how to switch strategies at runtime.

Strategy 1: swap the outgoing track

The simplest case. You send one video track. A button swaps its source between the camera and the screen. The transceiver, its mid, the SDP, and the negotiated codec never change; only the bytes feeding the encoder do.

This is the same replaceTrack mechanism used for flipping between front and rear cameras, applied to a camera/screen pair instead.

import { setupPeer } from '/src/salon/peer.js';

const pc = setupPeer();

// Start on the camera
const camStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
const camTrack = camStream.getVideoTracks()[0];
pc.addTrack(camTrack, camStream);        // one video transceiver, sendrecv
// ... negotiate once ...

// Later: switch the same transceiver to the screen
async function showScreen() {
  const screenStream = await navigator.mediaDevices.getDisplayMedia({ video: true });
  const screenTrack = screenStream.getVideoTracks()[0];
  screenTrack.contentHint = 'detail';    // crisp text, drop frames before resolution

  const sender = pc.getSenders().find(s => s.track?.kind === 'video');
  await sender.replaceTrack(screenTrack);

  // When the user hits the OS "Stop sharing" bar, fall back to the camera
  screenTrack.onended = () => sender.replaceTrack(camTrack);
}

No negotiationneeded fires. The remote peer's <video> element stays bound to the same incoming track; frames simply change content. The swap costs one frame of latency.

The constraint is structural: there is exactly one outgoing video, so the viewer can never see both at once. If the presenter wants their face visible while the slides show, this strategy cannot do it. That is not a tuning problem; it is the shape of the approach.

Two details make the swap reliable:

• Set contentHint on the screen track. 'detail' (or the older 'text') tells the encoder to keep resolution sharp and drop frame rate when the link congests. A screen of code at 5 fps reads fine; a screen of blurry code at 30 fps does not. Set 'motion' back on the camera track. See Display capture for the full hint table.
• Handle onended. The OS "Stop sharing" control ends the screen track behind your back. Wire onended to swap the camera back in, or the sender goes to a frozen last frame.

When the camera and screen differ wildly in resolution, the swap can still be free as long as the negotiated section can carry the larger size. If the screen needs a codec or resolution the section never agreed to, replaceTrack rejects and you are back to renegotiation, rare for video, but possible. Keep the original negotiation generous (offer a high max resolution) so either source fits.

The local preview during a swap

The swap changes what peers receive. It says nothing about what the presenter sees locally. Bind the local <video> element to whichever track is currently sending, or the presenter watches a stale source.

async function showScreen() {
  const screenStream = await navigator.mediaDevices.getDisplayMedia({ video: true });
  const screenTrack = screenStream.getVideoTracks()[0];
  screenTrack.contentHint = 'detail';

  const sender = pc.getSenders().find(s => s.track?.kind === 'video');
  await sender.replaceTrack(screenTrack);

  // Update the presenter's own preview to match what peers now get
  document.querySelector('#self').srcObject = screenStream;

  screenTrack.onended = () => {
    sender.replaceTrack(camTrack);
    document.querySelector('#self').srcObject = camStream;
  };
}

Keep the camera track alive while the screen is showing. Do not stop() it: you need it back the instant the screen ends, and re-acquiring through getUserMedia would stall and re-prompt. replaceTrack(null) is the way to pause sending without ending a track, but in a swap you always have the other source to hand, so swap straight to it.

When swap is the right call

Swap wins when the two sources are mutually exclusive in the UI anyway: a "share screen" button that replaces the speaker's tile rather than adding to it. It is the cheapest strategy on every axis: one encoder, one RTP stream, zero renegotiation, one frame of switch latency. Reach for anything heavier only when "both at once" is a hard requirement, not a nice-to-have.

Strategy 2: send both as two tracks

To show the camera and the screen at the same time, send two video tracks. Each gets its own transceiver, its own mid, its own m= line in the SDP. The viewer receives two independent videos and lays them out however it likes.

The first track is attached during the initial negotiation. The second is added later, when the presenter starts sharing, and adding it renegotiates.

The cross-link: adding a track renegotiates, swapping does not

This is the load-bearing distinction in this guide.

replaceTrack changes the source inside an existing sender. The set of media sections is unchanged, so no SDP changes, and negotiationneeded does not fire. That is why Strategy 1 is instant.

addTrack (or addTransceiver) changes the set of media sections. There is now an extra m= line the remote peer has never heard about. The SDP must change, so negotiationneeded fires and a full offer/answer round trip runs through your signaling channel before frames flow on the new section.

// Camera is already sending from the initial negotiation.
// Now start the screen as a SECOND video track.
async function addScreen() {
  const screenStream = await navigator.mediaDevices.getDisplayMedia({ video: true });
  const screenTrack = screenStream.getVideoTracks()[0];
  screenTrack.contentHint = 'detail';

  // This fires negotiationneeded: a new m=line is appended.
  const sender = pc.addTrack(screenTrack, screenStream);

  screenTrack.onended = () => {
    pc.removeTrack(sender);   // also renegotiates; the m=line goes inactive
  };
  return sender;
}

// The renegotiation itself runs in the negotiationneeded handler.
pc.onnegotiationneeded = async () => {
  await pc.setLocalDescription(await pc.createOffer());
  signaling.send({ type: 'offer', sdp: pc.localDescription });
};

Because adding the second track runs a round trip, there is a visible gap between "presenter clicks share" and "viewer sees the screen". Budget for it. On the manual-paste signaling mode in this project, that gap is however long it takes to copy a code between two humans, so for a two-track flow, prefer URL-hash or ntfy.sh signaling, which renegotiate without a human in the loop. See Signaling and renegotiation.

The receiver cannot tell the screen from the face

The viewer's ontrack now fires twice for video: once for the camera, once for the screen. Both events carry a MediaStreamTrack of kind: "video". Nothing in the track itself says which is the screen and which is the presenter. If you bind them to elements in arrival order, the layout flips at random between connections.

You need an out-of-band mapping. Three options, worst to best:

Track id matching. The sender knows its own track ids. Send a control message listing them.

// Sender, after addTrack:
signaling.send({ t: 'layout', screenTrackId: screenTrack.id, cameraTrackId: camTrack.id });

The receiver reads event.track.id and looks it up. This works but is brittle: the local track id is not guaranteed to survive to the remote side unchanged across all browsers, so do not lean on it alone.

Stream id grouping. Pass a distinct, named MediaStream to each addTrack. The stream id does travel in the SDP, and the receiver gets it in event.streams[0].id. Tell the receiver which stream id is the screen over your control channel.

// Sender
const screenGroup = new MediaStream([screenTrack]);
pc.addTrack(screenTrack, screenGroup);
signaling.send({ t: 'layout', screenStreamId: screenGroup.id });

// Receiver
pc.ontrack = (event) => {
  const streamId = event.streams[0]?.id;
  const slot = streamId === knownScreenStreamId ? '#screen' : '#camera';
  document.querySelector(slot).srcObject = event.streams[0];
};

Transceiver mid mapping. The most dependable key. The mid is stable for the life of the connection and identical on both peers. Fix the order of your transceivers, then address sections by mid.

// Sender: declare a fixed order up front, fill later
const camTx = pc.addTransceiver('video', { direction: 'sendrecv' }); // mid will be "0"
const screenTx = pc.addTransceiver('video', { direction: 'sendonly' }); // mid "1"
await camTx.sender.replaceTrack(camTrack);
// screen filled when sharing starts; mid already assigned after first negotiation
signaling.send({ t: 'layout', screenMid: screenTx.mid, cameraMid: camTx.mid });

// Receiver: map by the transceiver the track arrived on
pc.ontrack = (event) => {
  const mid = event.transceiver.mid;
  const slot = mid === knownScreenMid ? '#screen' : '#camera';
  document.querySelector(slot).srcObject = new MediaStream([event.track]);
};

The mid is the same handle that ties an SDP m= line to a transceiver, described in Tracks and streams. It does not change across renegotiations, so a screen that stops and restarts keeps its slot. That stability is why mid beats track and stream ids for this job.

A reusable section beats add-and-remove

Stopping a share with removeTrack and starting it again with addTrack works, but each one renegotiates, and the removed m= line never disappears; it goes inactive and a new section is appended on the next add. Over a long call with repeated sharing, the SDP grows a graveyard of dead sections.

The cleaner pattern reserves the screen section once, up front, with direction: 'sendonly' and no track. The first negotiation assigns its mid. Sharing then becomes a free replaceTrack on that reserved sender, and stopping becomes replaceTrack(null): neither renegotiates after the section exists.

// Reserve both sections during the initial negotiation
const camTx = pc.addTransceiver('video', { direction: 'sendrecv' });
const screenTx = pc.addTransceiver('video', { direction: 'sendonly' });
await camTx.sender.replaceTrack(camTrack);
// First offer/answer assigns camTx.mid = "0", screenTx.mid = "1"

// Start sharing: free, no renegotiation (the section already exists)
async function startShare() {
  const s = await navigator.mediaDevices.getDisplayMedia({ video: true });
  const t = s.getVideoTracks()[0];
  t.contentHint = 'detail';
  await screenTx.sender.replaceTrack(t);
  t.onended = stopShare;
}

// Stop sharing: free (pause the section without removing it)
async function stopShare() {
  await screenTx.sender.replaceTrack(null);   // section stays, goes silent
}

There is a tradeoff. A sendonly section reserved before any sharing means the SDP advertises a video section the viewer must be ready to receive from the first negotiation, even though it carries nothing yet. The viewer's ontrack may fire for an empty section, or not until the first frame, depending on the browser. Map by mid and the viewer can pre-create the empty #screen tile and reveal it when frames start. This front-loads the one renegotiation cost to call setup, where a round trip is cheap, and makes every subsequent share/stop instant.

Whichever key you choose, the rule is the same: send the camera/screen role explicitly over a control message. Never infer it from arrival order or from inspecting the track.

Costs of two tracks

Two encoders run in parallel. Two RTP streams share the same congestion-controlled link, so the screen and the camera compete for bandwidth on a tight connection. Tune each sender independently with setParameters: cap the camera's bitrate so the screen, which carries the detail that matters, keeps its share.

const camSender = pc.getSenders().find(s => s.track === camTrack);
const p = camSender.getParameters();
p.encodings[0].maxBitrate = 300_000;   // hold the camera down so the screen breathes
await camSender.setParameters(p);

Strategy 3: composite into one track

Draw the screen and the camera onto a single <canvas> every frame, screen filling the frame and camera inset in a corner, then capture the canvas as one video track and send that. The viewer receives a single, finished picture. The presenter owns the layout.

This is the only strategy that gives one outgoing track and both sources visible at once. It is also the only one that costs CPU continuously.

const screenStream = await navigator.mediaDevices.getDisplayMedia({ video: true });
const camStream = await navigator.mediaDevices.getUserMedia({ video: true });

// Off-screen <video> elements to draw from
const screenVid = Object.assign(document.createElement('video'),
  { srcObject: screenStream, muted: true });
const camVid = Object.assign(document.createElement('video'),
  { srcObject: camStream, muted: true });
await Promise.all([screenVid.play(), camVid.play()]);

const canvas = document.createElement('canvas');
canvas.width = 1280;
canvas.height = 720;
const ctx = canvas.getContext('2d');

let running = true;
function draw() {
  if (!running) return;
  // Screen fills the frame
  ctx.drawImage(screenVid, 0, 0, canvas.width, canvas.height);
  // Camera picture-in-picture, bottom-right
  const pw = canvas.width * 0.25;
  const ph = pw * (camVid.videoHeight / camVid.videoWidth || 0.5625);
  ctx.drawImage(camVid, canvas.width - pw - 16, canvas.height - ph - 16, pw, ph);
  requestAnimationFrame(draw);
}
draw();

// One track out of the canvas, capped frame rate for screen-style content
const composite = canvas.captureStream(15);   // 15 fps is plenty for slides
const compositeTrack = composite.getVideoTracks()[0];
compositeTrack.contentHint = 'detail';
pc.addTrack(compositeTrack, composite);

canvas.captureStream(fps) produces a MediaStreamTrack whose frames are whatever the canvas shows. The argument caps the frame rate; pass a low number for slide-heavy content to spend bandwidth on resolution instead of motion. The full canvas-capture mechanics (captureStream, requestFrame for on-demand capture, the cost model) are in Programmatic capture.

Three things keep a composite usable:

• Cap the frame rate to the content. A static slide does not need 30 fps. captureStream(10) or captureStream(15) halves or quarters the encode cost and the bandwidth, and the result still reads cleanly. Raise it only when the screen content actually moves.
• Match canvas size to the screen, not the camera. The screen is the part with small text. Size the canvas to the screen's resolution (read it from screenVid.videoWidth/Height after play) so you do not downscale the readable content to fit a camera-shaped frame.
• Stop the RAF loop on teardown. The draw loop runs forever otherwise. Set running = false and stop the source tracks when sharing ends, or you burn CPU on a canvas nobody sees.

The composite has one outgoing video section, so the viewer needs no role mapping: there is only one video. Adding it still renegotiates once (it is an addTrack), but only once, and never again as the presenter rearranges the layout. Moving the camera inset, resizing it, swapping which source is large: all of that happens inside the draw loop and changes nothing on the wire.

Sizing the canvas to the screen

The screen is the part with one-pixel-wide text. Size the canvas to the screen's actual capture resolution, read after the video element starts, not to a guessed constant.

await screenVid.play();
canvas.width = screenVid.videoWidth;     // match the readable source exactly
canvas.height = screenVid.videoHeight;

Oversizing wastes encode cycles on pixels that carry no detail. Undersizing throws away the readability that justified a screen share in the first place. A canvas drawn at the screen's native size, captured, then encoded, preserves text down to the encoder's own limits, and the encoder, hinted with 'detail', protects resolution over frame rate.

The camera inset is the opposite case. It is a face, motion-tolerant, and small on screen. Draw it at a quarter width or less. Its source can be a low-resolution capture (getUserMedia({ video: { width: 320 } })) since it will be scaled down anyway: capturing the camera at 1080p only to draw it 200 px wide burns capture bandwidth for nothing.

Driving the loop by content, not by clock

requestAnimationFrame redraws on every display refresh, 60 times a second on most screens. For a slide that has not changed, that is 59 wasted redraws. Two ways to cut it:

The blunt cut: throttle the loop to the capture rate. There is no point drawing faster than captureStream(15) samples.

let last = 0;
function draw(now) {
  if (!running) return;
  requestAnimationFrame(draw);
  if (now - last < 1000 / 15) return;   // skip frames above 15 fps
  last = now;
  ctx.drawImage(screenVid, 0, 0, canvas.width, canvas.height);
  drawInset();
}
requestAnimationFrame(draw);

The precise cut: drive capture explicitly with track.requestFrame() and only redraw when a source actually changed. Create the stream with captureStream(0) so it captures nothing on its own, then push a frame when you draw one. This is the cheapest possible composite for mostly-static content. See Programmatic capture for the full requestFrame model.

const composite = canvas.captureStream(0);   // 0 = capture only on request
const compositeTrack = composite.getVideoTracks()[0];

function redraw() {
  ctx.drawImage(screenVid, 0, 0, canvas.width, canvas.height);
  drawInset();
  compositeTrack.requestFrame();   // emit exactly one frame
}
// Call redraw() when the screen changes or every N ms for the camera inset

A blended approach fits the camera-plus-screen case well: redraw the inset region on a slow timer for the moving face, and redraw the whole canvas only when the screen content changes. Most presentations are static screens with a live face in the corner, and that split spends frames where motion actually is.

Tearing down cleanly

The draw loop, the off-screen video elements, and both source streams all outlive the share unless you stop them. Leaking any of them holds the camera light on or pins a core.

function endComposite() {
  running = false;                              // stop the RAF loop
  compositeTrack.stop();                        // end the outgoing track
  screenStream.getTracks().forEach(t => t.stop());
  camStream.getTracks().forEach(t => t.stop()); // release the camera, light off
  screenVid.srcObject = camVid.srcObject = null;
}

// The OS "Stop sharing" bar ends the screen source out from under you
screenStream.getVideoTracks()[0].onended = endComposite;

Releasing the camera here is the right move only if the composite was the whole point of holding it. If the camera also feeds a separate plain call, clone it (see Tracks and streams) so stopping the composite's copy leaves the call's copy alive.

Side by side

Read the table as a decision: need both visible and a CPU-cheap, layout-flexible viewer? Two tracks. Need both visible but want the presenter to fix the layout and keep the wire simple? Composite. Only ever need one at a time? Swap, and never renegotiate at all.

Switching strategies at runtime

A real app may move between strategies as the call evolves. The moves are not symmetric: some are free, some cost a round trip.

• Swap → composite. You had one toggling track; now you want both visible. Build the canvas, replaceTrack the composite track onto the existing sender. No renegotiation: the section count is unchanged, you swapped one source for another (the canvas) on the same transceiver. This is the cleanest upgrade path: start as a swap, promote to a composite in place when the presenter wants their face shown alongside the screen.

  // Existing video sender is showing the camera. Promote to composite in place.
  const sender = pc.getSenders().find(s => s.track?.kind === 'video');
  await sender.replaceTrack(compositeTrack);   // no negotiationneeded
  

• Composite → swap. Reverse: replaceTrack the plain camera or screen track back onto the sender and stop the draw loop. Also free.

• Swap → two tracks. Adding the second source as its own section. Renegotiates. Worth it only when the viewer must control layout.

• Two tracks → composite. Remove the second track (renegotiates), composite both into the remaining section's source via replaceTrack. One renegotiation for the removal.

The pattern: anything that changes the number of sections renegotiates; anything that changes the content of an existing section is free. Build on that. Reserve your video section count early and move between swap and composite with replaceTrack whenever you can, falling back to a renegotiated two-track layout only when the viewer genuinely needs two independent videos.

Recap

• There is no single "send screen and camera" call. Pick among three strategies by what the viewer must see and who owns the layout.
• Swap sends one video and toggles its source with replaceTrack. No renegotiation, instant, but only one source at a time.
• Two tracks sends two video sections. Both visible, viewer lays them out, but adding the second renegotiates, and the receiver must be told which section is the screen.
• Composite draws both into a canvas and sends captureStream(). Both visible, presenter owns the layout, one section, at the cost of a per-frame draw loop.
• The cross-link to remember: replaceTrack swaps a source with no SDP change; addTrack/addTransceiver adds a section and forces a round trip. Section count changes renegotiate; section content changes are free.
• For two tracks, map roles by transceiver mid and send the role explicitly over a control message. Never infer screen-vs-camera from arrival order.
• Set contentHint = 'detail' and cap frame rate on screen content so bandwidth goes to readable text, not motion.
• You can promote a swap to a composite with replaceTrack alone: no renegotiation. Use that path before reaching for two tracks.

Going further

• Display capture: getDisplayMedia, contentHint, and handling the OS "Stop sharing" event.
• Capturing audio and video: getting the camera and microphone tracks this guide assumes.
• Tracks and streams: the transceiver/mid/SDP model, replaceTrack, and the negotiationneeded rules underpinning every strategy here.
• Programmatic capture: canvas.captureStream, requestFrame, and the cost model behind the composite.
• Signaling and renegotiation: how the offer/answer from a track add reaches the other peer, and why a human-paste channel is a poor fit for runtime renegotiation.

Troubleshooting

The screen never appears on the viewer's side. With two tracks, a renegotiation has to complete before the new section carries frames. Check that negotiationneeded fired and that the offer/answer round trip finished. On a manual-paste signaling channel the gap is human-paced: switch to URL-hash or ntfy.sh for runtime adds.

The camera and the screen are swapped on the viewer. You inferred the role from ontrack arrival order. Map by transceiver mid and send the screen's mid over a control message; arrival order is not stable.

Switching from camera to screen stalls or rejects. Either you removed and re-added the track (renegotiation) instead of replaceTrack, or the screen needs a resolution the section never negotiated. Keep the initial offer's max resolution generous, and use replaceTrack for same-kind swaps.

Shared text is blurry. The encoder is spending bandwidth on frame rate. Set contentHint = 'detail' on the screen track and cap the capture frame rate (getDisplayMedia frame-rate constraint, or captureStream(15) for a composite). See Display capture.

The composite pins a CPU core. The draw loop runs every frame at full canvas size. Cap captureStream frame rate, size the canvas to the screen rather than oversizing it, and stop the RAF loop (running = false) and the source tracks when sharing ends.

On a tight link the screen degrades when the camera is on. Two tracks share one congestion-controlled connection. Cap the camera sender's maxBitrate with setParameters so the screen keeps its share, or switch to a composite so a single encoder allocates bandwidth across one frame.

Frozen last frame after the user clicks the OS "Stop sharing" bar. The screen track ended and nothing replaced it. Wire screenTrack.onended to replaceTrack the camera back in (swap) or removeTrack the screen section (two tracks).