lib/webcam-capture

Webcam capture for the WebSocket transport.

getUserMedia frames are encoded in the page with WebCodecs (H.264, else VP8) and handed to a transport-supplied sender one encoded frame at a time; the server's virtual camera decodes them. Codecs earn their place empirically: the probe ranks candidates on the camera's own frames and rejects one whose output decodes to the wrong picture (PROBE_COLOR_TOLERANCE); after it, a frame offered to a busy encoder is dropped rather than queued, a frame the encoder sits on counts the same way (createLagGauge), and the share behind moves the uplink down the ladder (createEncodePace). Past the last codec, and with no WebCodecs at all, JPEG frames from a canvas: more bytes at the camera's own rate. The WebRTC transport instead attaches the track to a sendonly transceiver (lib/webrtc.js setWebcam).

Orientation is relayed with each encoded frame, never drawn into the pixels: read from VideoFrame rotation/flip where exposed, derived from the window orientation where not (Safari's sensor-fixed frames). VideoEncoder rejects a mid-stream orientation change, so a turn rebuilds the encoder; the JPEG rung relays nothing, drawImage bakes what the engine knows.

Frames come off the track through the first source the engine offers: MediaStreamTrackProcessor on the page (Chromium), the worker-only processor with the track transferred in (Safari 18+), or a <video> element sampled with requestVideoFrameCallback (Firefox and the rest). That last source feeds the ladder only when webcam_encoder names a codec -- an engine landing there can hold the camera rate on a software encoder at the cost of a whole core, which no probe can price -- so auto sends its samples to the JPEG rung. Encoding runs in a worker when the engine allows (a transferable track never touches the page; otherwise each frame is transferred), else on the page thread through the same sources.

Classes

WebcamCapture

Defined in: lib/webcam-capture.js:628

Camera uplink for the WebSocket transport: opens the camera, settles on a frame source and an encoder, and hands encoded frames to sendFrame.

Constructors

Constructor

new WebcamCapture(opts): WebcamCapture;

Defined in: lib/webcam-capture.js:630

Parameters

Returns

WebcamCapture

Properties

width

width: number;

Defined in: lib/webcam-capture.js:635

height

height: number;

Defined in: lib/webcam-capture.js:636

fps

fps: number;

Defined in: lib/webcam-capture.js:637

bitrate

bitrate: number;

Defined in: lib/webcam-capture.js:638

quality

quality: number;

Defined in: lib/webcam-capture.js:639

encoderPreference

encoderPreference: string;

Defined in: lib/webcam-capture.js:640

Accessors

active

Get Signature

get active(): boolean;

Defined in: lib/webcam-capture.js:680

Whether a capture is running.

Returns

boolean

codec

Get Signature

get codec(): string;

Defined in: lib/webcam-capture.js:685

Name of the codec frames are sent as (a candidate name, or mjpeg), or null.

Returns

string

Methods

start()

start(deviceId): Promise<void>;

Defined in: lib/webcam-capture.js:712

Opens the camera and starts sending. Failures are reported through onError rather than thrown, and a track that ends (device unplugged, permission revoked) stops the capture.

Parameters

Returns

Promise<void>

requestKeyframe()

requestKeyframe(): void;

Defined in: lib/webcam-capture.js:771

Makes the next frame a keyframe: the server lost its decoder reference or just started.

Returns

void

stop()

stop(): void;

Defined in: lib/webcam-capture.js:783

Stops the capture and releases the camera, encoder and workers; idempotent.

Returns

void

setWireProvider()

setWireProvider(provider): void;

Defined in: lib/webcam-capture.js:948

Supplies a fresh MessagePort to each encode worker this capture spawns, giving encoded frames a line to the transport that skips the page thread; the far end may post {needKeyframe: true} back to force a resync. Only the worker encoder uses it -- the page-thread rungs keep sendFrame.

Parameters

Returns

void

Interfaces

WebcamCaptureOptions

Defined in: lib/webcam-capture.js:607

Properties

sendFrame

sendFrame: (codecId, keyframe, bytes, rotation?, flip?) => void;

Defined in: lib/webcam-capture.js:608

Delivers one encoded frame. rotation (clockwise degrees) and flip (horizontal, applied after the rotation) make its pixels upright and are 0 and false when they already are.

Parameters

Returns

void

onStateChange?

optional onStateChange?: (active) => void;

Defined in: lib/webcam-capture.js:612

Called when capture starts and stops.

Parameters

Returns

void

onError?

optional onError?: (error) => void;

Defined in: lib/webcam-capture.js:613

Called with getUserMedia and encoder failures.

Parameters

Returns

void

canSend?

optional canSend?: () => boolean;

Defined in: lib/webcam-capture.js:614

Returning false skips a frame (backpressure).

Returns

boolean

width?

optional width?: number;

Defined in: lib/webcam-capture.js:615

Capture width hint, 1280 by default.

height?

optional height?: number;

Defined in: lib/webcam-capture.js:616

Capture height hint, 720 by default.

fps?

optional fps?: number;

Defined in: lib/webcam-capture.js:617

Frame rate hint and send cadence cap, 30 by default.

bitrate?

optional bitrate?: number;

Defined in: lib/webcam-capture.js:618

Encoder bitrate in bits per second, 2500000 by default.

quality?

optional quality?: number;

Defined in: lib/webcam-capture.js:619

JPEG quality on the fallback rung, 0.8 by default.

encoderPreference?

optional encoderPreference?: string;

Defined in: lib/webcam-capture.js:620

A WEBCAM_ENCODER_PREFERENCES value (the webcam_encoder setting); auto by default.

Variables

WEBCAM_CODEC_MJPEG

const WEBCAM_CODEC_MJPEG: 0 = 0;

Defined in: lib/webcam-capture.js:38

Codec id of independent JPEG frames, as the server's webcam module numbers them.

WEBCAM_CODEC_H264

const WEBCAM_CODEC_H264: 1 = 1;

Defined in: lib/webcam-capture.js:40

Codec id of H.264 Annex B frames.

WEBCAM_CODEC_VP8

const WEBCAM_CODEC_VP8: 2 = 2;

Defined in: lib/webcam-capture.js:42

Codec id of VP8 frames.

WEBCAM_CODEC_VP9

const WEBCAM_CODEC_VP9: 3 = 3;

Defined in: lib/webcam-capture.js:44

Codec id of VP9 frames.

WEBCAM_CODEC_AV1

const WEBCAM_CODEC_AV1: 4 = 4;

Defined in: lib/webcam-capture.js:46

Codec id of AV1 temporal units.

WEBCAM_CODEC_H265

const WEBCAM_CODEC_H265: 5 = 5;

Defined in: lib/webcam-capture.js:48

Codec id of H.265 Annex B frames.

WEBCAM_ENCODER_PREFERENCES

const WEBCAM_ENCODER_PREFERENCES: string[];

Defined in: lib/webcam-capture.js:84

webcam_encoder values: auto = the ladder on MediaStreamTrackProcessor sources and JPEG on the <video> rung, a codec name = that codec alone everywhere (JPEG still the floor), mjpeg = JPEG everywhere.

PACE_MIN_SAMPLES

const PACE_MIN_SAMPLES: 60 = 60;

Defined in: lib/webcam-capture.js:87

Frames the encode pace is measured over before it can be believed.

PACE_BEHIND_RATIO

const PACE_BEHIND_RATIO: number;

Defined in: lib/webcam-capture.js:89

Share of offered frames the encoder may drop before it counts as too slow.

PACE_LAG_INTERVALS

const PACE_LAG_INTERVALS: 15 = 15;

Defined in: lib/webcam-capture.js:96

Capture intervals the oldest unanswered frame may age before the encoder counts as behind: some encoders hold frames in a pipeline encodeQueueSize never shows (Firefox H.264 runs tens of seconds stale at a queue of two). Half a second at 30 fps.

PROBE_COLOR_TOLERANCE

const PROBE_COLOR_TOLERANCE: 48 = 48;

Defined in: lib/webcam-capture.js:103

Region-mean color error between a probe frame and its own decoded output, past which the candidate encodes the wrong picture (some Firefox GPU stacks hand their encoder false chroma). Honest lossy encoding stays under a third of this.

Functions

createEncodePace()

function createEncodePace(): object;

Defined in: lib/webcam-capture.js:111

Share of offered frames the encoder was behind for, measured on live camera frames: the signal that a codec is too slow.

Returns

object

note

note: (arg0) => void;

Parameters

Returns

void

tooSlow

tooSlow: () => boolean;

Returns

boolean

behindRatio

behindRatio: () => number;

Returns

number

reset

reset: () => void;

Returns

void

createLagGauge()

function createLagGauge(fps): object;

Defined in: lib/webcam-capture.js:150

Staleness of the oldest frame sent to the encoder and not yet answered by a chunk; answering settles everything up to its timestamp, so an encoder that quietly discards inputs is not held to them. Stringified into the encode worker: keep it self-contained apart from PACE_LAG_INTERVALS.

Parameters

Returns

object

budgetMs

budgetMs: number;

sent

sent: (arg0, arg1) => void;

Parameters

Returns

void

answered

answered: (arg0) => void;

Parameters

Returns

void

lagMs

lagMs: (arg0) => number;

Parameters

Returns

number

reset

reset: () => void;

Returns

void

closeFrame()

function closeFrame(frame): void;

Defined in: lib/webcam-capture.js:185

Closes a VideoFrame; the <video> element the JPEG rung hands over has nothing to close.

Parameters

Returns

void

deriveRotation()

function deriveRotation(): number;

Defined in: lib/webcam-capture.js:210

Clockwise rotation that makes a sensor-orientation frame upright, from the current window orientation.

Returns

number

Degrees, a multiple of 90.

canDeriveOrientation()

function canDeriveOrientation(): boolean;

Defined in: lib/webcam-capture.js:219

Only mobile WebKit derives orientation: the one engine with a worker-only MediaStreamTrackProcessor and sensor-fixed frames carrying no transform. The worker source proves the engine, window.orientation the viewport.

Returns

boolean