selkies-wr-core

The WebRTC streaming core: one bundled peer connection carrying the video and audio the server encodes, the webcam and microphone uplinks on sendonly transceivers, and a data channel for everything else.

Client to server on the data channel, as text: SETTINGS,{json} (the persisted settings on connect and the passthrough settings later), r,WxH (stream resolution), s,DPI, vb,kbps, ab,bps, _arg_fps,N, _crf,N, _rc,mode, _ebc,bool, cmd,command, kr (release every key), cr (cache-only clipboard fetch), REQUEST_CLIPBOARD, START_VIDEO / STOP_VIDEO (this peer's feed), SET_NATIVE_CURSOR_RENDERING,0|1, the _f,fps and _l,ms client metrics, _stats_video,{json} when the server asked for raw reports, and the chunked clipboard transfer of lib/clipboard-worker-bridge.js. Server to client arrives through the WebRTCClient callbacks: the settings payload, clipboard-msg* messages, cursor and display-config updates, stats, and system actions (reload, mk_access,0|1, command_error,text, auth_success,{json} / role_update,{json}, resolution,WxH).

The page hash selects the role: none is the controller, #shared a strict viewer, #playerN a viewer with gamepad slot N, and #display2-<position> the secondary display page, which streams its own region of the extended desktop and keeps its per-display settings under _display2 keys. Signaling scopes controller and slot uniqueness per display id, the server runs one pipeline per display, and the position rides the connect metadata.

Contract with the dashboards. Globals published on window: selkiesLogs (capped log ring buffers), fps, network_stats, gpu_stats, system_stats, currentAudioBufferSize, manualResolution, enable_resize, streamResolutionDiverged, webrtcInput, and every server setting as window[key]. Window messages handled (same origin): setScaleLocally, resetResolutionToWindow, setManualResolution, setUseCssScaling, settings, command, pipelineControl, gamepadControl, clipboardUpdateFromUI, clipboardImageUpdate, audioDeviceSelected, requestFullscreen, setSynth, showVirtualKeyboard, setAntiAliasing, setUseBrowserCursors, touchinput:trackpad, touchinput:touch, plus the requestFileUpload DOM event. Window messages posted: sidebarButtonStatusUpdate, pipelineStatusUpdate, effectiveCursorState, serverSettings, clipboardContentUpdate, fileUpload warnings, trackpadModeUpdate, clientRoleUpdate, toggleDashboard, toggleTouchGamepad. Flags read: window.__selkiesModeSwitching (a mode switch in progress suppresses alerts and recovery reloads), window.__selkiesAuthProbe (re-presents the login after an auth drop), window.clipboard_enabled.

Functions

InitUI()

function InitUI(): void;

Defined in: selkies-wr-core.js:115

Injects the stylesheet for the video container, overlay and status bar.

Returns

void

webrtc()

function webrtc(): object;

Defined in: selkies-wr-core.js:224

Builds the WebRTC core.

Returns

object

initialize builds the DOM, connects signaling and opens the peer connection; cleanup tears the session down and resets every session-scoped value.

initialize

initialize: () => void;

Returns

void

cleanup

cleanup: () => void;

Returns

void

dcMessageBudget()

function dcMessageBudget(): number;

Defined in: selkies-wr-core.js:347

Per-message budget on the data channel: the negotiated SCTP maximum message size (the minimum of both ends) where the browser exposes it, else the 256 KiB pre-negotiation standard, capped at 1 MiB to bound per-message buffering, less 512 bytes for the message prefix.

Returns

number

safeSetItem()

function safeSetItem(key, value): void;

Defined in: selkies-wr-core.js:439

Writes a key, degrading a full or unavailable store to a warning.

Parameters

Returns

void

getFloatParam()

function getFloatParam(key, default_value): any;

Defined in: selkies-wr-core.js:475

Like getIntParam but keeps fractions: range-bounded settings can be floats.

Parameters

Returns

any

applyTimestamp()

function applyTimestamp(msg): string;

Defined in: selkies-wr-core.js:520

Prefixes a log line with the wall-clock time.

Parameters

Returns

string

alignResolution()

function alignResolution(num): number;

Defined in: selkies-wr-core.js:527

Rounds a dimension down to the encoder alignment: 2 (YUV 4:2:0 chroma), or 16 when forced.

Parameters

Returns

number

applyEffectiveCursorSetting()

function applyEffectiveCursorSetting(): void;

Defined in: selkies-wr-core.js:539

Applies the effective cursor-rendering setting: the user preference, or browser cursors forced on when this page is a secondary display or the primary while a secondary is connected, since the server-drawn overlay tracks only one capture region. The dashboard is told the value in force, so its toggle reflects the override.

Returns

void

playStream()

function playStream(): void;

Defined in: selkies-wr-core.js:554

Starts playback after the user's gesture and takes the wake lock.

Returns

void

requestWakeLock()

function requestWakeLock(): Promise<void>;

Defined in: selkies-wr-core.js:562

Keeps the screen awake while streaming; a no-op when held or where the API is absent.

Returns

Promise<void>

releaseWakeLock()

function releaseWakeLock(): Promise<void>;

Defined in: selkies-wr-core.js:581

Releases the wake lock if held.

Returns

Promise<void>

clearResumeWatchdog()

function clearResumeWatchdog(): void;

Defined in: selkies-wr-core.js:596

Cancels the resume watchdog.

Returns

void

armResumeWatchdog()

function armResumeWatchdog(): void;

Defined in: selkies-wr-core.js:611

Verifies that frames follow a START_VIDEO. The resume is one message on a data channel that can close at that very moment, and a lost one is answered with nothing: the peer would stay subscribed to a feed nobody encodes. The element's playback clock is the signal WebRTC has, so the check is whether it advanced past the mark taken here.

Returns

void

checkResumed()

function checkResumed(mark): void;

Defined in: selkies-wr-core.js:626

Watchdog tick: resends START_VIDEO while the playback clock has not moved past mark, up to RESUME_WATCHDOG_MAX_ATTEMPTS, then reloads to reconnect unless a fatal verdict or a mode switch forbids it. A tab hidden again stands the watchdog down: the visibility path owns that state. Each attempt also replays the element, since one the browser paused while the tab was away plays nothing however much RTP arrives.

Parameters

Returns

void

handleVisibilityChange()

function handleVisibilityChange(): Promise<void>;

Defined in: selkies-wr-core.js:661

Pauses this peer's video feed while the tab is hidden and resumes it on show, re-acquiring the wake lock the browser dropped.

A hidden tab's rendering is throttled anyway, so its encode and bandwidth are waste. STOP_VIDEO and START_VIDEO gate only this peer's RTP sender, and the shared capture stops once every consumer is paused, so viewer pages send them too. The pause is deferred because a navigating document reports hidden just before it unloads and timers never fire in an unloading document, so only a genuine tab-hide sends it. Recovery on resume is the server's IDR (plus PLI); the client sends no keyframe requests, and the resume watchdog verifies frames follow.

Returns

Promise<void>

applyOutputDevice()

function applyOutputDevice(): Promise<void>;

Defined in: selkies-wr-core.js:701

Routes audio to the preferred output device: the video element carries the bundled audio track, so setSinkId on it moves the audio sink.

Returns

Promise<void>

updateStatusDisplay()

function updateStatusDisplay(): void;

Defined in: selkies-wr-core.js:720

Shows the sentence-cased status (the internal value stays lower-case for comparisons); once connected, hides it and shows the play button if playback still needs a gesture.

Returns

void

updateVideoImageRendering()

function updateVideoImageRendering(): void;

Defined in: selkies-wr-core.js:736

Picks the video's image-rendering: pixelated with anti-aliasing off or at 1:1, auto (smoothed) when CSS-scaled above 1 dpr.

Returns

void

sanitizeAndStoreSettings()

function sanitizeAndStoreSettings(serverSettings): object;

Defined in: selkies-wr-core.js:780

Applies the server's settings payload to the runtime and reconciles the user's stored overrides against it.

Every value is applied to window[key]; only a genuine user override is persisted. A server value with no stored override is not written to localStorage, so a later server-side change can still be re-pushed, and a locked value is never written into the user's key, where it would masquerade as their pick after an unlock. The override is read under the key the dashboard writes (HiDPI stores as useCssScaling), or an unlocked operator value would win forever. An unlocked operator override with no stored pick is reported back as a change, so it is applied for real: window state alone leaves runtime consumers on their defaults. Ranged settings are parsed as floats: "0.5" read as an int is 0, out of range, and wiped on every connect. Plain values (audio_channels, app_terminal) configure pipelines rather than preferences and stay runtime-only.

Parameters

Returns

object

Corrections the server has to be told about.

sendClientPersistedSettings()

function sendClientPersistedSettings(): void;

Defined in: selkies-wr-core.js:863

Sends the persisted settings as the session's initial SETTINGS payload.

Every display page sends its own: the server applies a payload to the display whose channel delivered it, so a secondary configures only its stream, its resolution still riding the resize message. Per-display keys carry a _display2 suffix and each display reads only its own variant. Manual dimensions are exact physical pixels and go raw, as on the resize path. The DPR-derived scaling_dpi is seeded into this first payload unless a stored pick was collected: without it the desktop comes up at the default DPI and the dashboard's correction a second later forces a second capture restart on every HiDPI connect.

Returns

void

applyManualStyle()

function applyManualStyle(
   targetWidth, 
   targetHeight, 
   scaleToFit
): void;

Defined in: selkies-wr-core.js:942

Sizes and centers the video element for a manual resolution; the exact size is centered too, or a larger viewport would pin the box top-left.

Parameters

Returns

void

resetToWindowResolution()

function resetToWindowResolution(targetWidth, targetHeight): void;

Defined in: selkies-wr-core.js:1002

Sizes the video element to the window: the buffer hint in physical pixels, the on-screen box in CSS pixels (styling it with physical pixels overflows the viewport by dpr squared on HiDPI displays).

Parameters

Returns

void

autoDeriveDpi()

function autoDeriveDpi(): number;

Defined in: selkies-wr-core.js:1034

Derives scaling_dpi from the local display scaling so remote fonts match local ones: dpr 1.5 is 144, 2 is 192. Snapping to the nearest stop puts a density the stops do not name (a 3.5x phone, a 133% desktop) on the closest one and clamps at both ends.

Returns

number

sendResolutionToServer()

function sendResolutionToServer(width, height): void;

Defined in: selkies-wr-core.js:1052

Requests a stream resolution with the r,WxH message.

A manual resolution is the exact framebuffer and is not multiplied by the device pixel ratio, or a HiDPI toggle would swing it between 1x and 2x; an auto resolution is CSS pixels times dpr unless CSS scaling is on. Both are capped at 4080 so a dpr-2 4K fullscreen never asks for a 7680-wide framebuffer.

Parameters

Returns

void

enableAutoResize()

function enableAutoResize(): void;

Defined in: selkies-wr-core.js:1076

Follows window resizes with stream resolution requests.

Returns

void

disableAutoResize()

function disableAutoResize(): void;

Defined in: selkies-wr-core.js:1081

Stops following window resizes.

Returns

void

resizeStart()

function resizeStart(): void;

Defined in: selkies-wr-core.js:1099

Debounces window resizes into handleResizeUI.

Returns

void

resizeEnd()

function resizeEnd(): void;

Defined in: selkies-wr-core.js:1108

Runs handleResizeUI once the resize has been quiet for rdelta.

Returns

void

handleResizeUI()

function handleResizeUI(): void;

Defined in: selkies-wr-core.js:1130

Auto-mode resize: requests the window's CSS-pixel size from the server (sendResolutionToServer applies the device pixel ratio) and restyles the element onto it; shared by the debounced resize tail and the reset-to-window path.

A manual preset applied while a debounce is pending is not overwritten when it fires. enable_resize false pins the primary's resolution server-side, so the resize the server ignores is neither requested nor restyled onto; a secondary's stays allowed. The CSS size is clamped so the physical request stays within the 4080 cap and the element box matches what the server realizes.

Returns

void

watchDevicePixelRatio()

function watchDevicePixelRatio(): void;

Defined in: selkies-wr-core.js:1152

Re-runs the auto-resize path when the device pixel ratio changes: a window dragged to a monitor of another density, or an OS scaling change, fires no resize event, and the stream would stay at the old density until the next one. A matchMedia resolution query is one-shot at a given dppx, so it is re-armed after each change.

Returns

void

loadLastSessionSettings()

function loadLastSessionSettings(): void;

Defined in: selkies-wr-core.js:1179

Restores the last session's resolution and desktop DPI on connect.

The DPI goes through the server's idempotent set_dpi path, the same one the initial SETTINGS seed takes, so whichever lands first wins. Persisted trackpad mode re-asserts cursor compositing (touch has no hover cursor). A manual-mode secondary display reports its size on connect because a secondary lays out from what it reports; a pinned primary (enable_resize false) is styled to the window but keeps the server's resolution.

Returns

void

postSidebarButtonUpdate()

function postSidebarButtonUpdate(): void;

Defined in: selkies-wr-core.js:1206

Posts sidebarButtonStatusUpdate with the state of every pipeline toggle.

Returns

void

startWebcamCapture()

function startWebcamCapture(): Promise<void>;

Defined in: selkies-wr-core.js:1226

Starts the webcam uplink: the camera track rides the sendonly video transceiver the server reserved in the bundled SDP (the mirror of the microphone), so the browser's own encoder produces the H.264 or VP8 the server's virtual camera decodes, with RTP congestion control and no data-channel framing.

Returns

Promise<void>

stopWebcamCapture()

function stopWebcamCapture(): void;

Defined in: selkies-wr-core.js:1254

Stops the webcam uplink and reports the toggle state.

Returns

void

toggleGamepadConnection()

function toggleGamepadConnection(): boolean;

Defined in: selkies-wr-core.js:1268

Applies the gamepad toggle to the manager; a shared page always polls.

Returns

boolean

Whether polling is on.

handleMessage()

function handleMessage(event): void;

Defined in: selkies-wr-core.js:1297

Handles a same-origin dashboard window message; the module docblock lists the types. A shared page ignores the resolution, command and clipboard cases: a viewer never drives resolution policy, never reaches the server's command execution path and never writes its clipboard.

Parameters

Returns

void

handleSettingsMessage()

function handleSettingsMessage(settings, fromServer?): void;

Defined in: selkies-wr-core.js:1605

Applies a settings payload from the dashboard or the server.

A server-authored payload (the locked and overridden values replayed on every connect) is applied to the runtime but never written to the user's own keys, where it would outlive the lock and masquerade as their pick; only a dashboard-authored payload persists. Settings with no dedicated data-channel opcode ride a SETTINGS passthrough the server applies through handle_update_settings; displayPosition among them moves a secondary page to another side of the primary, and the primary ignores it.

Parameters

Returns

void

sendRespectiveRCvalue()

function sendRespectiveRCvalue(newMode): void;

Defined in: selkies-wr-core.js:1714

Re-sends the parameter the new rate-control mode reads: the bitrate for CBR, the CRF for CRF.

Parameters

Returns

void

enableStatWatch()

function enableStatWatch(): void;

Defined in: selkies-wr-core.js:1748

Starts the once-a-second stats loop: the essentials are published on window (fps, network_stats, currentAudioBufferSize) for the dashboards, the full connectionStat stays readable here, and enableWebrtcStatics streams the raw reports to the server as _stats_video.

A tick whose predecessor still awaits getStats() is skipped, since overlapping ticks would double-update the byte baselines, and the time window is re-anchored only on success, alongside those baselines, so both cover the same interval. The bandwidth reported is the received throughput (video plus audio), matching the WebSocket server's stat: availableReceiveBandwidth is only the congestion-control estimate and reads far below the real rate on a relay. The audio-buffer gauge is a proxy: the de-jitter depth over the 20 ms Opus frame approximates the frames buffered ahead of playout, since browser-managed audio exposes no frame count. The audio concealment counters (NetEQ) are the RED acceptance metric.

Returns

void

handleWindowFocus()

function handleWindowFocus(): Promise<void>;

Defined in: selkies-wr-core.js:1858

Releases every key server-side and, on Chromium, reads the local clipboard: Firefox and WebKit raise a paste prompt on every focus read, so there the read is driven only by the paste gesture handlers.

Returns

Promise<void>

handleWindowBlur()

function handleWindowBlur(): void;

Defined in: selkies-wr-core.js:1867

Releases every key server-side so none sticks across the blur.

Returns

void

setupKeyBoardAssisstant()

function setupKeyBoardAssisstant(): void;

Defined in: selkies-wr-core.js:1876

Forwards the control keys mobile keyboards emit as keydown on the off-screen assist input (Enter, Backspace); typed characters are handled by the Input class's own listener on the element.

Returns

void

notifyClipboardImageSkip()

function notifyClipboardImageSkip(reason, code): void;

Defined in: selkies-wr-core.js:1905

Tells the dashboard why a clipboard-image upload was skipped, in the fileUpload warning channel transfer warnings use.

Parameters

Returns

void

notifyClipboardImageWriteFailed()

function notifyClipboardImageWriteFailed(error): void;

Defined in: selkies-wr-core.js:1920

Tells the dashboard that a server image never reached the local clipboard. The panel shows nothing of an inbound image but this notice, so a write the browser refuses would otherwise read as the feature not working at all.

Parameters

Returns

void

sendClipboardData()

function sendClipboardData(
   data, 
   mimeType?, 
   onSkip?
): Promise<void>;

Defined in: selkies-wr-core.js:1946

Sends clipboard content to the server in chunks.

Uses the chunked transfer of lib/clipboard-worker-bridge.js, the same wire protocol as the WebSocket core, over the data channel with a drain gate: a multi-MB burst overflows the SCTP send buffer and Chromium closes the channel, taking the session with it. Raw chunks are sized so their base64 fits the data-channel message budget. Only a completed transfer marks the content synced, so a failure leaves it re-sendable.

Parameters

Returns

Promise<void>

handleClipboardData()

function handleClipboardData(msg): Promise<{
  isMultipart: boolean;
  mimeType: string;
  content: string | ClipboardItem;
}>;

Defined in: selkies-wr-core.js:2013

Decodes a server clipboard message, assembling multipart transfers.

Parameters

Returns

Promise<{ isMultipart: boolean; mimeType: string; content: string | ClipboardItem; }>

content is null while a multipart transfer is in progress, on failure, and for images on insecure origins, which have no ClipboardItem.

send()

function send(data): void;

Defined in: selkies-wr-core.js:2248

Strict viewers send nothing until collaboration is granted.

Parameters

Returns

void

applyRtcConfigAndConnect()

function applyRtcConfigAndConnect(config): void;

Defined in: selkies-wr-core.js:2702

Applies an RTC configuration and opens the connection. Shared by the fetched and the fallback configuration, so a failed TURN fetch still connects: the data channel delivers the server settings, and without it the dashboard never renders its controls or the transport toggle.

Parameters

Returns

void

References

webrtc

Re-exports webrtc