Expand description
pcmflux: PulseAudio/PipeWire audio capture with Opus encoding, plus mic-uplink playback, as a Rust library with an optional Python bridge (the audio sibling of pixelflux).
The capture thread pulls S16LE PCM fragments from a PulseAudio record stream,
reassembles them into fixed-size Opus frames, encodes them (mono/stereo via the
single-stream encoder, 5.1/7.1 surround via the multistream API), optionally wraps them
in RFC 2198 RED framing — redundant copies of recent frames, so a client on a lossy
transport rebuilds a dropped packet from the next one it receives instead of stalling
for a retransmit — and hands each frame to a delivery thread that runs the
AudioSink off the audio path, so a slow or blocked consumer can never stall the
PulseAudio pump. The playback path mirrors this in reverse: it decodes the Opus mic
uplink and writes PCM into a virtual sink.
The public API is AudioCapture and AudioPlayback, configured by
AudioCaptureSettings and AudioPlaybackSettings (whose Default is the
configuration the Python classes start with, and whose validate is the one place a
setting is refused), delivering AudioFrames to an AudioSink, reporting through
State, last_error, and Error, and swept by stop_all. The python feature
adds the pcmflux extension module (python.rs), thin wrappers over these types that
read settings by attribute name, export a frame through the buffer protocol, call the
capture callback on the delivery thread, and release the GIL around every join.
Concurrency design (the invariants below are load-bearing):
- A lifecycle mutex serializes joining/reassigning the capture thread.
- A single stop_state atomic is the one source of truth (0 = running, -1 = external stop, a positive value = self-stop recorded under the issuing thread’s tid — the delivery thread’s for a sink-issued stop). The external -1 is stored INSIDE that lock immediately before join, so a stop can never be lost between observing a live thread and asking it to stop. A re-entrant self-start undoes only its own self-stop via one compare-exchange, so a racing external stop is never clobbered.
- The PulseAudio mainloop is pumped with a bounded ~20ms timeout, so a stop is observed within ~20ms even if the audio source delivers no data (is wedged).
- Joining the capture thread transitively joins the delivery thread, so a stop must never wait for a lock the sink’s in-flight call may need: the Python bridge releases the GIL around every join for that reason, and a Rust sink owes its callers the same.
- A sink may itself call stop/start; the sink runs on the delivery thread, so that re-entrant case is detected via the delivery thread’s OS tid (the capture thread’s tid is checked too) and short-circuits without joining — a join from inside the sink would cycle (stopper joins capture, capture joins delivery, delivery is the stopper).
Modules§
- ogg_
sink 🔒 - Ogg Opus fan-out on a Unix socket: every consumer that connects receives the stream’s
OpusHeadandOpusTagspages, then one page per encoded packet, so the socket reads as a standard Ogg Opus stream (ffmpeg -i unix:<path>, the pixelflux recorder). - red 🔒
- RFC 2198 redundant audio as a client’s microphone sends it: where each block of a payload
sits, before anything is decoded. Nothing here reaches outside the module, so a fuzz build
(
fuzz/) compiles it on its own.
Macros§
- elog 🔒
- Non-panicking
eprintln!replacement — the stderr sibling ofplog!. - plog 🔒
- Non-panicking
println!replacement that swallows write errors (e.g. EPIPE) instead of panicking, so a broken output pipe can’t unwind the capture thread or a callback.
Structs§
- Audio
Capture - A capture: one PulseAudio record stream encoded to Opus and delivered to an
AudioSink. Owns theSharedlifecycle state;start/stop/update_audio_bitrate/set_stereo_companiondrive it, andis_capturing/state/last_errorreport it. Dropping the handle stops the capture. - Audio
Capture Settings - Capture/encode configuration, read once by
AudioCapture::start.Defaultis the configuration the PythonAudioCaptureSettingsstarts with: the system default source, 48 kHz stereo at 128 kbps VBR in 20 ms frames, the silence gate on, the audio header emitted, no RED, no output socket. - Audio
Frame - One encoded frame as the capture delivers it: the two-byte header and RED framing
followed by the Opus packet, or the raw packet on a capture that omits the header, with
the
ptsit starts at and the channels its Opus carries. - Audio
Playback - A mic-uplink playback, symmetric to
AudioCapture: the same lifecycle protocol, but a PulseAudio playback stream instead of a record stream, and a bounded drop-oldest queue fed bywrite/write_redinstead of a sink. The caller never holds the stream — only the playback thread touches it — so a close-versus-inflight-write use-after-free is structurally impossible here. Dropping the handle stops the playback. - Audio
Playback Settings - Mic-playback configuration, read once by
AudioPlayback::start, so the playback thread owns an immutable copy for the run’s lifetime.Defaultis the configuration the PythonAudioPlaybackSettingsstarts with, matching the client mic wire: theinputsink, S16LE mono at 24 kHz, a 40 ms sink buffer, and a queue bound of 96000 bytes (about 2 s). - Buffer
Pool 🔒 - Recycles outgoing frame buffers from dropped
AudioFrames back to the capture thread, so the steady-state emit path allocates nothing. - Delivery
Ring 🔒 - Bounded, drop-oldest hand-off from the capture thread to the delivery thread that runs the sink, so a slow or blocked sink can never stall the PulseAudio pump.
- Delivery
Thread 🔒 - Owns the delivery thread for the lifetime of one capture run and tears it down on
Drop, so teardown also happens when the capture thread UNWINDS. - Inner 🔒
- Shared state for one capture (or playback) run: the lifecycle state machine plus the
per-frame settings mirrors the worker reads on the hot path — all lock-free except the
rarely touched
last_error. - Mono
Opus 🔒 - Owning wrapper over a raw
OpusEncoder(the mono/stereo single-stream encoder). - Multi
Opus 🔒 - Owning wrapper over a raw
OpusMSEncoder(the surround multistream encoder). - OpusDec 🔒
- Owning wrapper over a raw
OpusDecoder. - Opus
Playback 🔒Decoder - Turns the mic uplink back into PCM: the client always sends the mic as Opus, so
every inbound packet must be decoded before it can be queued for the virtual sink. It
decodes one packet to interleaved S16LE PCM, reusing a scratch buffer across calls to
stay off the per-decode allocation path. Lives behind a
MutexonPbSharedand is driven fromwrite/write_red. - PaCapture
Session 🔒 - Own one whole capture run end to end on a dedicated thread — connect PulseAudio,
encode, and deliver — until stopped. It runs on its own thread because the PulseAudio
mainloop must be pumped continuously and independently of the caller: sharing the
caller’s thread would tie capture cadence to it and let any stall starve the audio.
The body handed to
spawn_worker. - PaPlayback
Session 🔒 - One PulseAudio session for playback: mainloop, context, and the playback stream, all
recreated together on reconnect — the mirror of
PaCaptureSession, with the same drop-order requirement (stream first, then its context, then the mainloop). - PbShared 🔒
- Per-
AudioPlaybackshared handle — the mirror ofSharedfor the playback path. - Play
Queue 🔒 - Bounded, drop-oldest byte queue for the mic-PCM handoff into the virtual “input” sink — the whole of the playback path’s buffering in a single bound.
- Pool
Taker 🔒 - Sole-consumer view over the shared
BufferPoolfor the capture thread. - RunState 🔒
- Per-run encode/deliver state, living on the capture thread’s stack for the lifetime of one capture. Holds the encoder, the frame-reassembly buffers, the outgoing buffer recycler, the RED redundancy history, and the running debug-log counters.
- Shared 🔒
- Per-
AudioCaptureshared handle: the lock-freeInnerstate plus the lifecycle-locked join handle for the capture thread. - Standing
Depth 🔒 - The playback queue’s standing depth, and when to cut it.
Enums§
- Error
- Why a call was refused: the one error type of the crate, which the Python bridge maps
onto
ValueErrorandRuntimeError. - PcmEncoder 🔒
- One encode surface over both Opus APIs: the single-stream C encoder for mono/stereo, and the multistream one for 6/8-channel surround.
- Session
Open 🔒Error - Why a session failed to open: drives the retry policy of the caller.
- State
- Lifecycle phase of a capture or playback:
Idle(no run yet, or the last run stopped cleanly),Starting(worker spawned, first PulseAudio session not yet up),Running(connected, or reconnecting mid-run, with no stop pending), orFailed(the last run ended in error;last_errorsays why). A run that was stopped readsIdleagain even while its thread is still winding down.
Constants§
- MAX_
OPUS_ 🔒PACKET - Max Opus output packet size in bytes per stream; sizes the emit buffer pool.
- OPUS_
BITRATE_ 🔒MAX - OPUS_
BITRATE_ 🔒MIN - Opus target-bitrate bounds (bits/s); the initial
opus_bitrateand every live update are clamped into this range so the encoder never sees a value it would reject. - OPUS_
COMPLEXITY_ 🔒MAX - Highest Opus encoder complexity.
- PUMP_
TIMEOUT_ 🔒US - PulseAudio mainloop pump timeout (~20 ms) — upper bound on how long a pending stop can go unobserved even when the audio source delivers nothing.
- RED_
BLOCK_ 🔒PT - RED block payload type.
- RED_
MAX_ 🔒LEN - Max RED block length — 10-bit ceiling in bytes.
- RED_
MAX_ 🔒OFFSET - Max RED timestamp offset — 14-bit ceiling in 48 kHz samples from the primary.
- RED_
PREFIX_ 🔒MAX - Worst-case byte length of the WS frame prefix (tag + n_red + pts + headers + payloads).
- STOP_
EXTERNAL 🔒 stop_statesentinel: authoritative external stop (-1). Any positive value instead is the OS tid of a capture thread that self-stopped from its callback.- STOP_
NONE 🔒 stop_statesentinel: no stop pending (running).- ST_
FAILED 🔒 start_state: the last run ended in error (startup or mid-run);last_errorsays why.- ST_IDLE 🔒
start_state: no worker has run yet, or the last run stopped cleanly.- ST_
RUNNING 🔒 start_statehandshake: the hot loop is running.- ST_
STARTING 🔒 start_statehandshake: startup in progress.- VERSION
- The crate version compiled in, for a host that names the pcmflux it links.
- WS_
QUIET 🔒 - Bit of a header-carrying capture’s second byte that marks where the stream pauses by the
sender’s choice. Alone, as
[0x01, WS_QUIET]with no Opus, it is sent where a sound ends (and when the session drops after sound): a player plays out what it holds at once rather than waiting for more, and running dry after it is not a late delivery. It is set as well on the silent frame sent right behind that mark, whose Opus carries the end of the sound the encoder held back, so a player appends that frame to the sound rather than taking it for a new one. Never part ofn_red, which fits below it (RED_MAX_DISTANCE).
Statics§
- PLAYBACK_
REGISTRY 🔒 - Process-wide registry of live playbacks, swept by
stop_allas captures are. HoldsWeakreferences so it keeps nothing alive on its own. - REGISTRY 🔒
- Process-wide registry of live captures, swept by
stop_all. HoldsWeakreferences so it keeps nothing alive on its own.
Traits§
- Audio
Sink - Where a capture delivers its encoded frames:
frameis called on the capture’s delivery thread, off the audio path, once per frame, in order. A closureFn(AudioFrame) + Send + Syncis a sink. The sink may call the capture’sstoporstartfrom insideframe; such a re-entrant call is detected by thread and never joins.
Functions§
- await_
start 🔒 - Startup handshake shared by the capture and playback starts: waits up to ~2 s for the
worker this call spawned (
spawned) to publishRUNNINGorFAILED. A worker stillSTARTINGwhen the window closes counts as started — its retry ladder legitimately runs longer than this — and a failure after that is exposed throughstate/last_errorinstead. OnFAILED,join_failed_starttears down only that thread (identity-checked, sparing a concurrent winner) and the worker’s recorded error is returned asError::Start. - capture_
run 🔒 - check_
opus_ 🔒sample_ rate - Opus codecs only run at 8, 12, 16, 24, or 48 kHz; anything else would fail encoder or decoder creation on the worker, so it is rejected up front.
- checked_
device_ 🔒name - A device name as libpulse gets it:
Noneand the empty string both mean the system default, and an interior NUL is refused. Shared by both settings’ validation. - downmix_
stereo 🔒 - Fold one surround frame (
surround_channel_maporder) to interleaved stereo,dstholding the frame’s samples per channel twice: each side takes its front channel at unity and the center and its surround channels at -3 dB (ITU-R BS.775, the LFE left out), saturating rather than wrapping, as a browser’s own speaker downmix of the surround stream sounds. - gettid 🔒
- Returns the calling thread’s OS tid (
gettidsyscall) so a stop/start issued from inside the sink can detect it is on one of the run’s own threads and avoid self-joining. - join_
failed_ 🔒start - Failed-start teardown: stop and join the thread
spawnedby THIS start attempt, but only if it still owns the slot. - join_
held_ 🔒worker - Join the worker
slotholds, having set the external stop INSIDE the caller’s lock immediately beforejoin()— the set-before-join ordering the lost-stop invariant rests on — and clearcapture_tidso a later start sees no stale owner. Every stop path ends here. - multiopus_
layout 🔒 - Chromium’s multistream-Opus surround layout for a channel count, as
(streams, coupled, mapping);Nonefor anything but 5.1 (6) or 7.1 (8). - pa_
capture_ 🔒session_ open - Open a capture session: mainloop + context + record stream driven to
Readyon the bounded pump, honoringstop_pendingat every turn. A NAMED device is validated by an introspect probe on every call (an async connect_record would not fail synchronously on a bad name, and on reconnect this is also what notices the device reappearing after an outage).device_was_presentdistinguishes initial bring-up from reconnect: a named device missing at startup is probably a misconfiguration, so it is reported asDeviceNotFoundand the caller spends only a short window on it; the same device vanishing mid-run (PulseAudio/PipeWire restart kills every source) is transient and must be retried or audio never comes back. - pa_
playback_ 🔒session_ open - Open a playback session: mainloop + context + playback stream driven to
Readyon the bounded pump, honoringstop_pendingat every turn. The mirror ofpa_capture_session_open. - playback_
registry 🔒 - Lazily initialize and return the playback registry.
- playback_
run 🔒 - Drive one whole mic-playback run on the playback thread. The body handed to
spawn_worker; the mirror ofcapture_runfor the uplink. - pump 🔒
- Run one bounded iteration of a PulseAudio standard mainloop:
prepare(timeout_us)→poll→dispatch. Returnsfalseif any stage errors. - registry 🔒
- Lazily initialize and return the capture registry.
- settings_
error 🔒 - Build the
Error::Settingsfor a rejected settings field. - spawn_
worker 🔒 - Locked takeover + spawn of a worker thread, shared by the capture and playback
starts. Returns the new thread’s id, or
Noneif the spawn failed. - stop_
all - Stop and join every live capture and playback, so no worker thread outlives the
program’s teardown: snapshots the two
Weakregistries into strong references (skipping any already dropped), then hands each tostop_and_join. The Python bridge registers it onatexit. - stop_
and_ 🔒join join_held_workerbehind the lifecycle lock, for the stop paths that want nothing else from it. A lock poisoned by an unrelated panic is taken anyway: it must never be what leaves a worker thread running into process exit.- surround_
channel_ 🔒map - The speaker positions the surround encoder takes its input channels as, in order, as a
PulseAudio channel map: Chromium’s 5.1 and 7.1 orders (left, right, center, LFE, then the
surround pair, and at 7.1 the back pair before the side one), which its
multiopusdecoder hands to the output layout. The record stream asks for this map, so the sound server remixes the source’s own layout into it. Without one, libpulse gives a stream its AIFF default, which at six channels names front-left-of-center, front-right-of-center, and rear-center instead of the surround pair and LFE, and at eight has no map at all, so the stream cannot even be created.Nonefor mono and stereo, whose defaults are right. - valid_
opus_ 🔒duration - True if
msis a valid Opus frame duration (2.5, 5, 10, 20, 40, or 60 ms). - write_
ws_ 🔒prefix_ into - Emit the RFC 2198 RED framing prefix into
bufin-place and return its length, so the encoder can serialize the Opus packet directly after it with no scratch buffer and no per-frame allocation. The runtime counterpart ofbuild_ws_body.
Type Aliases§
- Frame
Queue 🔒 - Backing store for the delivery ring:
Some(queue)while open,Noneonce closed sopopwakes and returnsNonefor a clean shutdown.