PipelineBridge
A bridge to asynchronously pass data between Media and the RTC pipeline.
maxsize selects the buffering policy: depth 1 is latest-wins (video wants the freshest frame), a deeper bound acts as a short drop-oldest FIFO (audio wants continuity so a brief consumer stall doesn't silently drop samples).
A drop happens upstream of RTP: no sequence number is spent, so the
receiver sees no gap and never asks for a keyframe, while every delta
frame behind the drop references a picture it never received. With
request_keyframe bound the bridge keeps the wire decodable the way the
websockets relay does. A frame that names what it predicts from is
dropped with a word to the encoder (invalidate_reference), which then
predicts past it, and only the frames predicting from a dropped one are
held back, so the stream resumes on the next frame without a keyframe. A
frame that names nothing closes a gate that holds delta frames back, a
keyframe is asked for until one arrives and reopens it, and a queued
keyframe is never evicted by a delta frame. A gate no keyframe answers
within GATE_TIMEOUT_S reopens on its own.
Attributes
attributedropped= 0attributeinvalidated= 0Functions
constructor__init__(maxsize=1, request_keyframe=None, clock=time.monotonic, invalidate_reference=None) -> NoneInitializes the bridge.
Source Code
def __init__(self, maxsize: int = 1,
request_keyframe: Optional[Callable[[], None]] = None,
clock: Callable[[], float] = time.monotonic,
invalidate_reference: Optional[Callable[[int], None]] = None) -> None:
"""Initializes the bridge.
Args:
maxsize: Queue depth; 1 means latest-wins, larger is drop-oldest.
request_keyframe: Asks the display's encoder for a keyframe, on
the loop thread, at most once per IDR_REQUEST_FLOOR_S while
the gate is closed. None leaves every item ungated: audio
samples are self-contained.
clock: Monotonic time source.
invalidate_reference: Tells the display's encoder a frame id was
dropped, so the frames after it stop predicting from it. None
gates every drop behind a keyframe.
Attributes:
dropped: Items discarded since construction, evicted for a newer
one or held back behind a closed gate. None of them spent a
sequence number, so they appear in no loss statistic on either
side; this counter is what separates a lagging sender from a
lossy link.
invalidated: The share of those the encoder was told to predict
past, the rest being frames that predicted from one already
dropped and so needed no word of their own.
"""
self._queue: asyncio.Queue = asyncio.Queue(maxsize=maxsize)
self._request_keyframe = request_keyframe
self._invalidate = invalidate_reference
self._clock = clock
self._queued_keyframe = False
self._gated_at: Optional[float] = None
self._last_request: Optional[float] = None
# Frame ids dropped recently, which nothing delivered may predict from.
self._lost: deque = deque(maxlen=LOST_FRAME_MEMORY)
self.dropped = 0
self.invalidated = 0parammaxsizeint= 1Queue depth; 1 means latest-wins, larger is drop-oldest.
paramrequest_keyframeOptional[Callable[[], None]]= NoneAsks the display's encoder for a keyframe, on the loop thread, at most once per IDR_REQUEST_FLOOR_S while the gate is closed. None leaves every item ungated: audio samples are self-contained.
paramclockCallable[[], float]= time.monotonicMonotonic time source.
paraminvalidate_referenceOptional[Callable[[int], None]]= NoneTells the display's encoder a frame id was dropped, so the frames after it stop predicting from it. None gates every drop behind a keyframe.
Returns
Nonefuncset_data(data, keyframe=True) -> NoneEnqueue an item, dropping the oldest one when the queue is full.
Synchronous, no lock: the checks and the put have no await, so the single-threaded loop runs them without interleaving (all access is on the loop thread). A full queue means the consumer is lagging.
Source Code
def set_data(self, data: Any, keyframe: bool = True) -> None:
"""Enqueue an item, dropping the oldest one when the queue is full.
Synchronous, no lock: the checks and the put have no await, so the
single-threaded loop runs them without interleaving (all access is on
the loop thread). A full queue means the consumer is lagging.
Args:
data: The item.
keyframe: Whether the item decodes on its own. A delta frame needs
the frame it predicts from on the wire: one that names it is
held back only while that frame was dropped here, and one that
does not is held back behind any drop until the next keyframe.
"""
queue = self._queue
if self._request_keyframe is None:
if queue.full():
queue.get_nowait()
self.dropped += 1
queue.put_nowait(data)
return
if keyframe:
if queue.full():
queue.get_nowait()
self.dropped += 1
queue.put_nowait(data)
self._queued_keyframe = True
self._gated_at = None
self._lost.clear()
return
dependency = data.dependency if self._invalidate is not None else None
if dependency is not None:
frame_id, reference = dependency
if reference in self._lost:
self._lost.append(frame_id)
self.dropped += 1
return
if queue.full():
if self._queued_keyframe:
self._drop(data)
return
self._drop(queue.get_nowait())
if reference in self._lost:
self._lost.append(frame_id)
self.dropped += 1
return
queue.put_nowait(data)
self._queued_keyframe = False
return
now = self._clock()
if self._gated_at is not None:
if now - self._gated_at < GATE_TIMEOUT_S:
self.dropped += 1
self._ask(now)
return
logger.warning("Video bridge: no keyframe within %.1fs of a drop; "
"sending delta frames again", GATE_TIMEOUT_S)
self._gated_at = None
if queue.full():
self.dropped += 1
if not self._queued_keyframe:
queue.get_nowait()
self.dropped += 1
self._gated_at = now
self._ask(now)
return
queue.put_nowait(data)
self._queued_keyframe = FalseparamdataAnyThe item.
paramkeyframebool= TrueWhether the item decodes on its own. A delta frame needs the frame it predicts from on the wire: one that names it is held back only while that frame was dropped here, and one that does not is held back behind any drop until the next keyframe.
Returns
Nonefunc_ask(now) -> NoneSource Code
def _ask(self, now: float) -> None:
if self._last_request is not None and now - self._last_request < IDR_REQUEST_FLOOR_S:
return
self._last_request = now
self._request_keyframe()paramnowfloatReturns
Nonefunc_drop(item) -> NoneLet a frame go and tell the encoder, so nothing later predicts from it.
Source Code
def _drop(self, item: Any) -> None:
"""Let a frame go and tell the encoder, so nothing later predicts from it."""
self.dropped += 1
self.invalidated += 1
self._lost.append(item.dependency[0])
self._invalidate(item.dependency[0])paramitemAnyReturns
Nonefuncempty() -> boolSource Code
def empty(self) -> bool:
return self._queue.empty()Returns
boolfuncget_data() -> AnyWait until an item is available in the queue and return it.
Source Code
async def get_data(self) -> Any:
"""Wait until an item is available in the queue and return it."""
return await self._queue.get()Returns
typing.Any