selkies-wr-core

The WebRTC streaming core: one bundled peer connection carrying the video and audio the server encodes, the webcam and microphone uplinks on sendonly transceivers, and a data channel for everything else.

Client to server on the data channel, as text: SETTINGS,{json} (the persisted settings on connect and the passthrough settings later), r,WxH (stream resolution), s,DPI, vb,kbps, ab,bps, _arg_fps,N, _crf,N, _rc,mode, _ebc,bool, cmd,command, kr (release every key), cr (cache-only clipboard fetch), REQUEST_CLIPBOARD, START_VIDEO / STOP_VIDEO (this peer's feed), SET_NATIVE_CURSOR_RENDERING,0|1, vp,originX,originY,scaleX,scaleY (this page's stream box on the user's desktop, relayed to the other displays), the _f,fps and _l,ms client metrics, _stats_video,{json} when the server asked for raw reports, and the chunked clipboard transfer of lib/clipboard-worker-bridge.js. Server to client arrives through the WebRTCClient callbacks: the settings payload, clipboard-msg* messages, cursor and display-config updates, stats, and system actions (reload, mk_access,0|1, command_error,text, auth_success,{json} / role_update,{json}, resolution,WxH, video_declined,mime, capture_demand,<subject>,0|1).

The page hash selects the role: none is the controller, #shared a strict viewer, #playerN a viewer with gamepad slot N, and #display2-<position> the secondary display page, which streams its own region of the extended desktop and keeps its per-display settings under _display2 keys. Signaling scopes controller and slot uniqueness per display id, the server runs one pipeline per display, and the position rides the connect metadata.

Contract with the dashboards. Globals published on window: selkiesLogs (capped log ring buffers), fps, stream_info, stream_client and stream_stats (lib/stream-stats.js), currentAudioBufferSize, manualResolution, enable_resize, streamResolutionDiverged, webrtcInput, and every server setting as window[key]. Window messages handled (same origin): setScaleLocally, resetResolutionToWindow, setManualResolution, setUseCssScaling, settings, command, pipelineControl, gamepadControl, clipboardUpdateFromUI, clipboardImageUpdate, audioDeviceSelected, requestFullscreen, setSynth, showVirtualKeyboard, setAntiAliasing, setUseBrowserCursors, setRawPointerMotion, touchinput:trackpad, touchinput:touch, statsOpen, plus the requestFileUpload DOM event. Window messages posted: sidebarButtonStatusUpdate, pipelineStatusUpdate, effectiveCursorState, scalingDpiFollowed, serverSettings, clipboardContentUpdate, fileUpload warnings, trackpadModeUpdate, clientRoleUpdate, toggleDashboard, toggleTouchGamepad. Flags read: window.__selkiesModeSwitching (a mode switch in progress suppresses alerts and recovery reloads), window.__selkiesAuthProbe (re-presents the login after an auth drop), window.clipboard_enabled.

Functions

InitUI()

function InitUI(): void;

Defined in: selkies-wr-core.js:123

Injects the stylesheet for the video container, overlay and status bar.

Returns

void

webrtc()

function webrtc(): object;

Defined in: selkies-wr-core.js:232

Builds the WebRTC core.

Returns

object

initialize builds the DOM, connects signaling and opens the peer connection; cleanup tears the session down and resets every session-scoped value.

initialize

initialize: () => void;

Returns

void

cleanup

cleanup: () => void;

Returns

void

dcMessageBudget()

function dcMessageBudget(): number;

Defined in: selkies-wr-core.js:386

Per-message budget on the data channel: the negotiated SCTP maximum message size (the minimum of both ends) where the browser exposes it, else the 256 KiB pre-negotiation standard, capped at 1 MiB to bound per-message buffering, less 512 bytes for the message prefix.

Returns

number

reencodePngOffThread()

function reencodePngOffThread(blob): Promise<Blob>;

Defined in: selkies-wr-core.js:451

PNG-normalizes an image on the worker, which is where the decode and re-encode of a large one belong; the page's own canvas covers a worker that cannot do it.

Parameters

Returns

Promise<Blob>

safeSetItem()

function safeSetItem(key, value): void;

Defined in: selkies-wr-core.js:512

Writes a key, degrading a full or unavailable store to a warning.

Parameters

Returns

void

streamDensity()

function streamDensity(): number;

Defined in: selkies-wr-core.js:531

Stream pixels per CSS pixel this page requests and draws at (lib/stream-density.js).

Returns

number

followStreamDensity()

function followStreamDensity(): void;

Defined in: selkies-wr-core.js:547

Hands the density to the input layer and, on a secondary whose density moved (a HiDPI or UI-scaling change, or on X11 the primary's scale), requests the stream at it again.

Returns

void

getFloatParam()

function getFloatParam(key, default_value): any;

Defined in: selkies-wr-core.js:579

Like getIntParam but keeps fractions: range-bounded settings can be floats.

Parameters

Returns

any

applyTimestamp()

function applyTimestamp(msg): string;

Defined in: selkies-wr-core.js:624

Prefixes a log line with the wall-clock time.

Parameters

Returns

string

alignResolution()

function alignResolution(num): number;

Defined in: selkies-wr-core.js:631

Rounds a dimension down to the encoder alignment: 2 (YUV 4:2:0 chroma), or 16 when forced.

Parameters

Returns

number

applyEffectiveCursorSetting()

function applyEffectiveCursorSetting(): void;

Defined in: selkies-wr-core.js:643

Applies the effective cursor-rendering setting: the user preference, or browser cursors forced on when this page is a secondary display or the primary while a secondary is connected, since the server-drawn overlay tracks only one capture region. The dashboard is told the value in force, so its toggle reflects the override.

Returns

void

applyRawPointerMotion()

function applyRawPointerMotion(): void;

Defined in: selkies-wr-core.js:658

Applies the raw pointer motion setting to the input handler.

Returns

void

applyMacCmdAsCtrl()

function applyMacCmdAsCtrl(): void;

Defined in: selkies-wr-core.js:671

Applies the Command-as-Control setting to the input handler.

Returns

void

applyKeyboardShortcuts()

function applyKeyboardShortcuts(): void;

Defined in: selkies-wr-core.js:680

Applies the chord setting to the input handler.

Returns

void

resolvedRawPointerMotion()

function resolvedRawPointerMotion(serverSettings): boolean;

Defined in: selkies-wr-core.js:694

The raw_pointer_motion the deployment implies, off the shared ladder: a locked or operator value, else the client's stored pick, else the platform default (off on macOS). Resolved here because pointer lock is taken from the stream, whether or not a settings panel ever mounts.

Parameters

Returns

boolean

resolvedMacCmdAsCtrl()

function resolvedMacCmdAsCtrl(serverSettings): boolean;

Defined in: selkies-wr-core.js:706

The mac_cmd_as_ctrl the deployment implies, off the shared ladder. Resolved here for the reason raw pointer motion is: the keyboard is taken from the stream, whether or not a settings panel ever mounts.

Parameters

Returns

boolean

resolvedCssScaling()

function resolvedCssScaling(serverSettings): boolean;

Defined in: selkies-wr-core.js:722

The use_css_scaling the deployment implies, off the shared ladder: a locked or operator value, else the client's explicit pick, else CSS scaling whenever a resolution is configured (the sanitized payload has just published the server's manual mode and width on window). Resolved here rather than left to a settings panel, which a dashboard may mount only when it is opened; until then the session would stream pixel-perfect against a configuration that says otherwise, on every load.

Parameters

Returns

boolean

playStream()

function playStream(): void;

Defined in: selkies-wr-core.js:732

Starts playback after the user's gesture and takes the wake lock.

Returns

void

requestWakeLock()

function requestWakeLock(): Promise<void>;

Defined in: selkies-wr-core.js:740

Keeps the screen awake while streaming; a no-op when held or where the API is absent.

Returns

Promise<void>

releaseWakeLock()

function releaseWakeLock(): Promise<void>;

Defined in: selkies-wr-core.js:759

Releases the wake lock if held.

Returns

Promise<void>

clearResumeWatchdog()

function clearResumeWatchdog(): void;

Defined in: selkies-wr-core.js:774

Cancels the resume watchdog.

Returns

void

armResumeWatchdog()

function armResumeWatchdog(): void;

Defined in: selkies-wr-core.js:789

Verifies that frames follow a START_VIDEO. The resume is one message on a data channel that can close at that very moment, and a lost one is answered with nothing: the peer would stay subscribed to a feed nobody encodes. The element's playback clock is the signal WebRTC has, so the check is whether it advanced past the mark taken here.

Returns

void

checkResumed()

function checkResumed(mark): void;

Defined in: selkies-wr-core.js:805

Watchdog tick: resends START_VIDEO while the playback clock has not moved past mark, up to RESUME_WATCHDOG_MAX_ATTEMPTS, then reloads to reconnect unless a fatal verdict or a mode switch forbids it. A tab hidden again stands the watchdog down, the visibility path owning that state, and so does a stream the server declined, which no resend brings back. Each attempt also replays the element, since one the browser paused while the tab was away plays nothing however much RTP arrives.

Parameters

Returns

void

handleVisibilityChange()

function handleVisibilityChange(): Promise<void>;

Defined in: selkies-wr-core.js:840

Pauses this peer's video feed while the tab is hidden and resumes it on show, re-acquiring the wake lock the browser dropped.

A hidden tab's rendering is throttled anyway, so its encode and bandwidth are waste. STOP_VIDEO and START_VIDEO gate only this peer's RTP sender, and the shared capture stops once every consumer is paused, so viewer pages send them too. The pause is deferred because a navigating document reports hidden just before it unloads and timers never fire in an unloading document, so only a genuine tab-hide sends it. Recovery on resume is the server's IDR (plus PLI); the client sends no keyframe requests, and the resume watchdog verifies frames follow.

Returns

Promise<void>

applyOutputDevice()

function applyOutputDevice(): Promise<void>;

Defined in: selkies-wr-core.js:880

Routes audio to the preferred output device: the video element carries the bundled audio track, so setSinkId on it moves the audio sink.

Returns

Promise<void>

updateStatusDisplay()

function updateStatusDisplay(): void;

Defined in: selkies-wr-core.js:902

Shows the sentence-cased status (the internal value stays lower-case for comparisons); once connected, hides it and shows the play button if playback still needs a gesture, unless the page was told its video is declined, whose notice stays.

Returns

void

updateVideoImageRendering()

function updateVideoImageRendering(): void;

Defined in: selkies-wr-core.js:918

Picks the video's image-rendering: pixelated with anti-aliasing off or at 1:1, auto (smoothed) when CSS-scaled above 1 dpr.

Returns

void

sanitizeAndStoreSettings()

function sanitizeAndStoreSettings(serverSettings): object;

Defined in: selkies-wr-core.js:960

Applies the server's settings payload to the runtime and reconciles the user's stored overrides against it.

Every value is applied to window[key]; only a genuine user override is persisted. A server value with no stored override is not written to localStorage, so a later server-side change can still be re-pushed, and a locked value is never written into the user's key, where it would masquerade as their pick after an unlock. The override is read under the key the dashboard writes (HiDPI stores as useCssScaling), or an unlocked operator value would win forever. An unlocked operator override with no stored pick is reported back as a change, so it is applied for real: window state alone leaves runtime consumers on their defaults. Ranged settings are parsed as floats: "0.5" read as an int is 0, out of range, and wiped on every connect. Plain values (audio_channels) configure pipelines rather than preferences and stay runtime-only.

Parameters

Returns

object

Corrections the server has to be told about.

receiverTakesVp9FullColor()

function receiverTakesVp9FullColor(): boolean;

Defined in: selkies-wr-core.js:1031

Whether this engine's RTP receiver takes VP9 profile 1, the 4:4:4 profile.

Returns

boolean

settleFullColorSupport()

function settleFullColorSupport(): Promise<void>;

Defined in: selkies-wr-core.js:1051

Turns full color off where this engine cannot decode the codec's 4:4:4 profile over RTP.

Where the decoder has no 4:4:4 profile a full-color stream is not a heavier picture but no picture, so the setting is dropped rather than asked for. The receiver's own capabilities answer for VP9 (profile 1); the decoder probe answers for H.264 and H.265. Written to storage, which is what every payload and both dashboards read, so the toggle shows what the stream is.

Returns

Promise<void>

fullColorDecodable()

function fullColorDecodable(codec): Promise<boolean>;

Defined in: selkies-wr-core.js:1065

Whether this engine decodes the codec's 4:4:4 over WebRTC: the receiver's capabilities for VP9 (profile 1), the decoder probe for H.264 and H.265.

Parameters

Returns

Promise<boolean>

declineUndecodableFullColor()

function declineUndecodableFullColor(): Promise<void>;

Defined in: selkies-wr-core.js:1078

Turns a full color the server announced off again where this engine cannot decode the codec's 4:4:4 over WebRTC, so the stream comes back 4:2:0 on the same codec instead of arriving as one this browser paints nothing of. A locked setting cannot be turned off and is reported once.

Returns

Promise<void>

measuredDisplayScale()

function measuredDisplayScale(): number;

Defined in: selkies-wr-core.js:1112

This page's remote pixels per CSS pixel: its decoded buffer over the box drawing it, the ratio a neighbor scales a cross-display drag by (lib/stream-density.js). The requested density stands in until the realized buffer catches up with it.

Returns

number

applyManualStyle()

function applyManualStyle(
   targetWidth, 
   targetHeight, 
   scaleToFit
): void;

Defined in: selkies-wr-core.js:1216

Sizes and centers the video element for a manual resolution; the exact size is centered too, or a larger viewport would pin the box top-left. Exact is one stream pixel per device pixel, independent of the HiDPI flag, which a manual resolution does not read.

Parameters

Returns

void

resetToWindowResolution()

function resetToWindowResolution(targetWidth, targetHeight): void;

Defined in: selkies-wr-core.js:1279

Sizes the video element to the window: the buffer hint in physical pixels, the on-screen box in CSS pixels (styling it with physical pixels overflows the viewport by dpr squared on HiDPI displays).

Parameters

Returns

void

autoDeriveDpi()

function autoDeriveDpi(): number;

Defined in: selkies-wr-core.js:1308

Derives the default scaling_dpi (lib/stream-density.js): from the local display scaling, so remote fonts match local ones, or from a manual resolution, which is a framebuffer of its own that the local screen says nothing about. A stored pick overrides either.

Returns

number

effectiveScalingDpi()

function effectiveScalingDpi(): number;

Defined in: selkies-wr-core.js:1325

The DPI the desktop is asked for: 96 under CSS scaling, where the pick divides the requested resolution instead (lib/stream-density.js).

A manual resolution is the exact framebuffer either way -- a HiDPI toggle must not swing the size the operator asked for -- so there is nothing for the pick to divide there and it governs the desktop, HiDPI or not. Applying it once is not the double scaling that CSS scaling had: the request it would have divided is fixed.

Returns

number

pushScalingDpi()

function pushScalingDpi(): void;

Defined in: selkies-wr-core.js:1330

Sends the desktop the DPI in force.

Returns

void

followDerivedDpi()

function followDerivedDpi(reason): boolean;

Defined in: selkies-wr-core.js:1341

Re-derives scaling_dpi while it sits on its automatic default; a stored value is the dashboard's explicit pick and is left alone. The new value is posted as scalingDpiFollowed so the dashboards show it.

Parameters

Returns

boolean

Whether the derived value moved; the caller pushes it.

maybeFollowDpr()

function maybeFollowDpr(): void;

Defined in: selkies-wr-core.js:1360

Follows a live devicePixelRatio change while scaling_dpi sits on its automatic default, re-deriving and pushing it so the remote UI density matches the display the window is on. Called from both the resize handler and the matchMedia density watcher: an OS scaling change can surface as either, and emulated density changes fire only the resize.

Returns

void

sendResolutionToServer()

function sendResolutionToServer(width, height): void;

Defined in: selkies-wr-core.js:1378

Requests a stream resolution with the r,WxH message.

A manual resolution is the exact framebuffer and is not multiplied by the device pixel ratio, or a HiDPI toggle would swing it between 1x and 2x; an auto resolution is CSS pixels times dpr unless CSS scaling is on. Both are capped at 4080 so a dpr-2 4K fullscreen never asks for a 7680-wide framebuffer.

Parameters

Returns

void

enableAutoResize()

function enableAutoResize(): void;

Defined in: selkies-wr-core.js:1408

Follows window resizes with stream resolution requests.

Returns

void

disableAutoResize()

function disableAutoResize(): void;

Defined in: selkies-wr-core.js:1413

Stops following window resizes.

Returns

void

resizeStart()

function resizeStart(): void;

Defined in: selkies-wr-core.js:1431

Debounces window resizes into handleResizeUI.

Returns

void

resizeEnd()

function resizeEnd(): void;

Defined in: selkies-wr-core.js:1441

Runs handleResizeUI once the resize has been quiet for rdelta.

Returns

void

handleResizeUI()

function handleResizeUI(): void;

Defined in: selkies-wr-core.js:1463

Auto-mode resize: requests the window's CSS-pixel size from the server (sendResolutionToServer applies the device pixel ratio) and restyles the element onto it; shared by the debounced resize tail and the reset-to-window path.

A manual preset applied while a debounce is pending is not overwritten when it fires. enable_resize false pins the primary's resolution server-side, so the resize the server ignores is neither requested nor restyled onto; a secondary's stays allowed. The CSS size is clamped so the physical request stays within the 4080 cap and the element box matches what the server realizes.

Returns

void

watchDevicePixelRatio()

function watchDevicePixelRatio(): void;

Defined in: selkies-wr-core.js:1489

Re-runs the auto-resize path when the device pixel ratio changes: a window dragged to a monitor of another density, or an OS scaling change, fires no resize event, and the stream would stay at the old density until the next one. While scaling_dpi sits on its automatic default it is re-derived and pushed too, so the remote UI density follows the display the window is on. A matchMedia resolution query is one-shot at a given dppx, so it is re-armed after each change.

Returns

void

loadLastSessionSettings()

function loadLastSessionSettings(): void;

Defined in: selkies-wr-core.js:1520

Restores the last session's resolution and desktop DPI on connect.

The DPI goes through the server's idempotent set_dpi path, the same one the initial SETTINGS seed takes, so whichever lands first wins. Persisted trackpad mode re-asserts cursor compositing (touch has no hover cursor). A manual-mode secondary display reports its size on connect because a secondary lays out from what it reports; a pinned primary (enable_resize false) is styled to the window but keeps the server's resolution.

Returns

void

postSidebarButtonUpdate()

function postSidebarButtonUpdate(): void;

Defined in: selkies-wr-core.js:1547

Posts sidebarButtonStatusUpdate with the state of every pipeline toggle.

Returns

void

setMicrophone()

function setMicrophone(on, askedByServer?): void;

Defined in: selkies-wr-core.js:1566

Starts or stops the microphone uplink and reports the outcome to the sidebar.

Parameters

Returns

void

startWebcamCapture()

function startWebcamCapture(askedByServer?): Promise<void>;

Defined in: selkies-wr-core.js:1588

Starts the webcam uplink: the camera track rides the sendonly video transceiver the server reserved in the bundled SDP (the mirror of the microphone), so the browser's own encoder produces the H.264, VP8, VP9, H.265 or AV1 the server's virtual camera decodes, with RTP congestion control and no data-channel framing.

Parameters

Returns

Promise<void>

stopWebcamCapture()

function stopWebcamCapture(): void;

Defined in: selkies-wr-core.js:1619

Stops the webcam uplink and reports the toggle state.

Returns

void

toggleGamepadConnection()

function toggleGamepadConnection(): boolean;

Defined in: selkies-wr-core.js:1633

Applies the gamepad toggle to the manager; a shared page always polls.

Returns

boolean

Whether polling is on.

handleMessage()

function handleMessage(event): void;

Defined in: selkies-wr-core.js:1662

Handles a same-origin dashboard window message; the module docblock lists the types. A shared page ignores the resolution, command and clipboard cases: a viewer never drives resolution policy, never reaches the server's command execution path and never writes its clipboard.

Parameters

Returns

void

handleSettingsMessage()

function handleSettingsMessage(settings, fromServer?): void;

Defined in: selkies-wr-core.js:2010

Applies a settings payload from the dashboard or the server.

A server-authored payload (the locked and overridden values replayed on every connect) is applied to the runtime but never written to the user's own keys, where it would outlive the lock and masquerade as their pick; only a dashboard-authored payload persists. Settings with no dedicated data-channel opcode ride a SETTINGS passthrough the server applies through handle_update_settings; displayPosition among them moves a secondary page to another side of the primary, and the primary ignores it.

Parameters

Returns

void

sendRespectiveRCvalue()

function sendRespectiveRCvalue(newMode): void;

Defined in: selkies-wr-core.js:2153

Re-sends the parameter the new rate-control mode reads: the bitrate for CBR, the CRF for CRF.

Parameters

Returns

void

askDecodeCapable()

function askDecodeCapable(
   codec, 
   width, 
   height, 
   fps
): void;

Defined in: selkies-wr-core.js:2196

Whether the engine decodes the stream's configuration efficiently, which stands in for the decoder's name where the engine withholds it.

Parameters

Returns

void

sampleStreamStats()

function sampleStreamStats(
   stats, 
   rtt, 
   mbps
): void;

Defined in: selkies-wr-core.js:2218

One second of this page's figures for lib/stream-stats.js, from the same getStats() snapshot the stat watch took: rates are the change in a cumulative counter since the last tick, and the repair counters read from zero at the moment the stats opened.

Parameters

Returns

void

handleWindowFocus()

function handleWindowFocus(): Promise<void>;

Defined in: selkies-wr-core.js:2398

Releases every key server-side and, on Chromium, reads the local clipboard: Firefox and WebKit raise a paste prompt on every focus read, so there the read is driven only by the paste gesture handlers.

Returns

Promise<void>

handleWindowBlur()

function handleWindowBlur(): void;

Defined in: selkies-wr-core.js:2407

Releases every key server-side so none sticks across the blur.

Returns

void

notifyClipboardImageSkip()

function notifyClipboardImageSkip(reason, code): void;

Defined in: selkies-wr-core.js:2417

Tells the dashboard why a clipboard-image upload was skipped, in the fileUpload warning channel transfer warnings use.

Parameters

Returns

void

notifyClipboardImageWriteFailed()

function notifyClipboardImageWriteFailed(error): void;

Defined in: selkies-wr-core.js:2432

Tells the dashboard that a server image never reached the local clipboard. The panel shows nothing of an inbound image but this notice, so a write the browser refuses would otherwise read as the feature not working at all.

Parameters

Returns

void

sendClipboardData()

function sendClipboardData(
   data, 
   mimeType?, 
   onSkip?
): Promise<void>;

Defined in: selkies-wr-core.js:2458

Sends clipboard content to the server in chunks.

Uses the chunked transfer of lib/clipboard-worker-bridge.js, the same wire protocol as the WebSocket core, over the data channel with a drain gate: a multi-MB burst overflows the SCTP send buffer and Chromium closes the channel, taking the session with it. Raw chunks are sized so their base64 fits the data-channel message budget. Only a completed transfer marks the content synced, so a failure leaves it re-sendable.

Parameters

Returns

Promise<void>

handleClipboardData()

function handleClipboardData(msg): Promise<{
  isMultipart: boolean;
  mimeType: string;
  content: string | ClipboardItem;
}>;

Defined in: selkies-wr-core.js:2537

Decodes a server clipboard message, assembling multipart transfers.

Parameters

Returns

Promise<{ isMultipart: boolean; mimeType: string; content: string | ClipboardItem; }>

content is null while a multipart transfer is in progress, on failure, and for images on insecure origins, which have no ClipboardItem. preview carries a flavour set's text, which the item itself does not hand back synchronously.

send()

function send(data): void;

Defined in: selkies-wr-core.js:2803

Strict viewers send nothing until collaboration is granted.

Parameters

Returns

void

applyStartPolicy()

function applyStartPolicy(serverSettings): void;

Defined in: selkies-wr-core.js:3266

Applies the session's start policy from a peer connection's first server_settings payload: the *_on_start settings say which pipelines this page starts with. Only a session owner's primary display page is governed (a shared viewer cannot switch video or audio on, and a second display page exists to show its display), and a pipeline the user already toggled keeps that choice. The server registered this peer with its video and audio senders paused by the same policy, so video and audio off only mirror that here; the microphone and webcam start their uplinks; the gamepad policy seeds a toggle the browser has not persisted.

Parameters

Returns

void

applyRtcConfigAndConnect()

function applyRtcConfigAndConnect(config): void;

Defined in: selkies-wr-core.js:3417

Applies an RTC configuration and opens the connection. Shared by the fetched and the fallback configuration, so a failed TURN fetch still connects: the data channel delivers the server settings, and without it the dashboard never renders its controls or the transport toggle.

Parameters

Returns

void

References

webrtc

Re-exports webrtc