Struct ReferenceWindow 

pub struct ReferenceWindow {
    frames: VecDeque<(u16, u64, bool)>,
    capacity: usize,
    lends: bool,
    anchors: Vec<Option<(u16, u64, bool)>>,
    recent: VecDeque<(u16, u64, bool)>,
    next_pts: u64,
    age: KeyAge,
    key_pts: u64,
    frame_num_range: u64,
    acknowledged: bool,
    common: VecDeque<u64>,
    sent: VecDeque<u64>,
}

The reference frames a session’s decoder holds, in encode order, with the ones a client reported lost marked.

An encoder told to forget a frame leaves it and every frame after it out of its predictions, predicts the next frame from the newest frame before them, and codes a key frame when none is left. The window mirrors that, so the reference a frame carries on the wire is the one its bitstream uses, and it asks for the key frame where the encoder would. Frames are addressed by the capture’s frame id, which wraps; the timestamp the encoder is told to forget is the session’s own count of encoded frames, which does not.

An H.264 session also names how many values its frame_num takes before wrapping (set_frame_num_range). A decoder that never receives the frame carrying frame_num 0 sees a gap across that wrap, and FFmpeg’s H.264 decoder, which the browsers decode with on Linux, derives the picture order past such a gap a wrap short and drops the pictures after it until that order passes the last one shown (13 of a range of 16 after two frames lost), so a loss covering that frame is answered with a key frame rather than a prediction past it.

A window with_anchors also keeps long-term frames out of the buffer: the key frame is the first anchor and frames after it are marked on a schedule, and a loss older than every recent frame is predicted past from the newest anchor before it.

Where the consumers say which frames every one of them holds, or was sent (acknowledge), several of them share the stream and each recovers on its own: the newest anchor all of them hold is pinned, the schedule marking the other, so a loss of any depth is predicted past from a frame every consumer has; an anchor predicts from a frame they were all sent where that is near (settle), so each can decode it; and in H.264 the frame where frame_num wraps is an anchor too.

Fields

frames: VecDeque<(u16, u64, bool)>

capacity: usize

lends: bool

anchors: Vec<Option<(u16, u64, bool)>>

recent: VecDeque<(u16, u64, bool)>

next_pts: u64

age: KeyAge

key_pts: u64

frame_num_range: u64

acknowledged: bool

common: VecDeque<u64>

sent: VecDeque<u64>

Implementations

impl ReferenceWindow

pub fn new(capacity: u32) -> Self

pub fn with_anchors(capacity: u32, count: usize) -> Self

A window over a decoded picture buffer of capacity frames, count of them anchors (one or two).

pub fn lend_free_anchors(&mut self)

An anchor holding no frame, or a lost one, lends its share of the buffer to the recent frames, as NVENC’s AV1 encoder does with a long-term slot it holds nothing in.

fn recent_capacity(&self) -> usize

The recent frames the buffer holds: its share beside the anchors, and that of each anchor lending its own (lend_free_anchors).

pub fn anchored(&self) -> bool

Whether the window keeps anchors.

pub fn anchor_count(&self) -> usize

How many anchors the window keeps.

pub fn plan_anchor(&self, key: bool) -> Option<u8>

The anchor the next frame is marked into, if any: the first for a key frame or while none is held, and on the schedule after it the one that is empty, lost, or older, never the one pinned (acknowledge) where there are two, nor one every consumer was sent and is yet to hold (PENDING_PERIODS), nor a lone one they all hold while frames they were all sent go unheld (HOLD_LAG) or while it would predict from the one it marks over (reaches_back). Acknowledged, an H.264 frame where frame_num wraps is an anchor too.

fn reaches_back(&self) -> bool

Whether an anchor marked now would predict from an older one (settle): no recent frame held is one every consumer was sent, as while one of them lags, and the newest anchor is under HOLD_MAX frames old. A lone anchor so marked costs every consumer the bits of a prediction from an anchor period back, and the one it would mark over serves the one lagging as well.

fn anchor_period(&self) -> u64

How many frames apart the schedule marks anchors.

fn wraps_next(&self) -> bool

Whether the next frame carries H.264 frame_num 0, the consumers acknowledging.

pub fn set_acknowledged(&mut self)

The consumer says which frames its clients hold (acknowledge) from the first frame on.

pub fn acknowledge(&mut self, frame_id: u16, held: bool) -> bool

Note that every consumer holds frame_id, or where not held was sent it; false for a frame not remembered, one a consumer reported lost, or one from before the last key frame.

pub fn acknowledging(&self) -> bool

Whether the consumers acknowledge what they hold or were sent (set_acknowledged).

fn is_common(&self, pts: u64) -> bool

Whether every consumer holds the frame at pts: the key frame, which each is sent before any other, one acknowledged, or, acknowledged, an H.264 frame a consumer never leaves out (kept), so the anchor at the frame_num wrap stays pinned until a newer one is held.

fn kept(&self, pts: u64) -> bool

Whether the frame at pts is one a consumer sends every client of an H.264 stream it sends any of (pixelflux’s consumers do): where frame_num wraps, which a decoder cannot be predicted past without.

fn is_shared(&self, pts: u64) -> bool

Whether every consumer was sent the frame at pts, as every one holds a common one.

pub fn predicts_from_shared(&self) -> bool

Whether every consumer was sent the frame the next one predicts from, asked before that one is recorded: a lone anchor marked over the one it predicts from lets that one go.

fn pinned_anchor(&self) -> Option<usize>

The anchor holding the newest frame every consumer holds, which the schedule leaves.

pub fn settle(&mut self) -> Option<Invalidation>

Before an anchor: where the newest frame held is not one every consumer was sent, forget the frames after the newest that is, so the anchor predicts from it and every consumer can decode it. The invalidation that does it, for the encoder; None where none is needed, no frame held is one they were all sent, or that one lies further back than SETTLE_SLACK or, of two anchors, SETTLE_RECENT allows.

pub fn forget_stale(&mut self) -> Option<Invalidation>

Before an H.264 frame: where the newest frame held is a recent one further back than the buffer holds recent frames, forget the frames after the newest anchor every consumer was sent (acknowledged: the pinned one, a newer one they were all sent, or the one at the frame_num wrap, which a consumer never leaves out), else every recent one, so the frame predicts from an anchor. A decoder that fills a frame_num gap with pictures of its own (FFmpeg’s, which Chromium decodes in software with, and Firefox and WebKit on Linux) has let that recent frame go by then for a client that missed the frames since (anchors among them, which the recent frames leave out), and fails a frame that names it (Chromium’s does). The invalidation, for the encoder; None where none is needed.

pub fn predicting_anchor(&self) -> Option<u8>

The anchor the next frame predicts from, where the newest frame held is an anchor; the encoder names it, since NVENC’s H.264 fallback takes the first anchor otherwise.

pub fn anchor_slot(&self, pts: u64) -> Option<u8>

The anchor holding the frame at pts, where one does, lost or not: an encoder that writes its own headers lists it long-term under that index while the decoder holds it.

pub fn set_frame_num_range(&mut self, range: u32)

How many values the stream’s frame_num takes before it wraps; 0 for a codec without that counter. A range other than the one before is logged (frame_num_range_line).

pub fn reset(&mut self)

Forget every frame held, for a stream the session restarts with a key frame of its own.

pub fn set_capacity(&mut self, capacity: u32)

The decoded picture buffer the session has now; frames past it are let go. The anchors take their share of it.

pub fn next_pts(&self) -> u64

The timestamp the next frame is encoded with.

fn anchor_frames(&self) -> impl Iterator<Item = (u16, u64, bool)> + '_

The anchors held, in no order.

pub fn has_reference(&self) -> bool

Whether a frame not coded as a key frame has a reference left to predict from.

pub fn newest_valid(&self) -> Option<(u16, u64)>

The newest frame the client still has, as its id and timestamp: the one the next frame predicts from.

pub fn held(&self) -> impl Iterator<Item = (u16, u64, bool)> + '_

The frames the decoder holds, oldest first: each frame’s id, timestamp, and whether a client reported it lost.

pub fn key_pts(&self) -> u64

The timestamp of the last key frame, where the counts a codec keeps restart.

pub fn record(&mut self, frame_id: u16, key: bool) -> Reference

Record the frame just encoded with next_pts and answer what it predicted from.

pub fn record_marked( &mut self, frame_id: u16, key: bool, anchor: Option<u8>, ) -> Reference

Record the frame just encoded with next_pts, marked into anchor where the session marked it (plan_anchor), and answer what it predicted from.

pub fn retract(&mut self, forget: impl FnOnce(u64) -> bool) -> bool

Take back the frame just recorded, which the session codes again in its place under the same id: it is left out of the predictions as a frame the client lost is, and forget tells the encoder its timestamp. False, with the window as it was, where it was a key frame, nothing older is left to predict from, it carries an H.264 frame_num 0, which the decoder has to see (set_frame_num_range), or forget refuses.

pub fn invalidate(&mut self, frame_id: u16) -> Invalidation

Leave frame_id and every frame encoded after it out of the references. A report for a picture already removed by an earlier invalidation leaves the recovered chain intact. The bounded history dates those reports across capture-id gaps and wrap; a loss older than the history keeps the conservative key-frame fallback.

fn invalidate_recent(&mut self, frame_id: u16) -> Invalidation

Invalidate a session with only recent references.

fn invalidate_anchored(&mut self, frame_id: u16) -> Invalidation

invalidate where anchors are kept: a loss older than every recent frame is dated by the frames remembered and predicted past from an anchor before it, where one is left.