Module pipeline 

Frame-processing policy shared by the X11 and Wayland backends. Frame-processing policy shared by the two capture backends. It lives in its own module for one reason: the Wayland path (dmabuf, compositor damage) and the X11 path (host-ARGB, stripe-hash damage) capture pixels in completely different ways, but a viewer must never be able to tell which one produced a frame — a paint-over refresh or a recovery keyframe has to behave identically either way. Keeping the decision logic here, source-agnostic, is what guarantees it.

Structs

EncoderQuality

What an encoder says about the frames it codes, which the cleanup reads: the quality index its rate control last coded a frame at, where it reports one (FrameEncoder::last_quality), the bytes of that frame, where it reports them too (FrameEncoder::last_size), whether it holds a frame at a quantizer it is asked for (FrameEncoder::holds_quantizer), whether a change of its constant quality re-opens it on a key frame (FrameEncoder::reopens_on_quality), whether its constant-rate cleanup ends on a key frame (FrameEncoder::cleans_up_with_key), where it holds a band of a frame at that quantizer (FrameEncoder::band_size), the bytes of its last held frame, where it measures a frame’s reconstruction against its source when asked (FrameEncoder::measures), the last measurement, and where it took its last frame back for coming out past its cap and coded it again (FrameEncoder::refit), how far that frame’s held band then reached, or 0 for a frame of its rate control’s.

HwFrameDecision

Outcome of the full-frame send decision produced by decide_hw_fullframe.

X11Pipeline

Everything the X11 host-ARGB path has to remember between frames.

Enums

Cleanup

What the cleanup policy decided for one region’s frame.

Convergence

Where a rate control that the cleanup runs through stands after its last frame, of bytes at quality index last, against the paint-over quantizer and budget bytes a frame.

Damage

How much of a frame changed since the frame before it, as far as its capture can tell.

Constants

BAND_BUDGETS 🔒

CLEANUP_FRAME_S 🔒

Seconds of a link at the cleanup’s rate a cleanup’s frame may take to cross it (cleanup_cap): the session’s buffer at 8 fps, under the backpressure gate’s allowance.

CLEANUP_RATE_MAX 🔒

The most a cleanup runs at, in multiples of the session’s rate (cleanup_rate): a sweep at 8 fps and 500 kbps over a fast path codes bands of two to six budgets.

CONVERGED_SHARE 🔒

A constant-rate cleanup that runs through the rate control keeps the frames flowing until the rate control codes the region at the paint-over quantizer or finer in a frame under this share of the frame budget: it has nothing left to refine.

CONVERGE_S 🔒

Seconds a constant-rate cleanup through the rate control flows at most.

DAMAGE_BAND_ROWS 🔒

Rows per band of the content hash a full-frame X11 session reads its damage from: one band is under SMALL_AREA of a screen from 720 rows up, so a caret or a clock reads as the small change it is.

EMPTY_S 🔒

Seconds of such frames after which a constant-rate cleanup through x264 ends: x264 keeps its quantizer on a screen that does not change, and at 0.1 Mbit/s refines one in frames as far as 2.4 s apart. A session that holds no quantizer and measures nothing ends the same way once its frames at its coarsest have kept the size of the one before as long (SAME_FRAME_SHARE).

EMPTY_SLICE_BYTES 🔒

Bytes a slice that refines nothing comes to at most: a frame of x264’s four slices skipping every block takes 61 to 66 at 720p and 1080p with the wire header, a third of the budget at 0.1 Mbit/s, so never a small frame.

FIRST_BAND 🔒

Where the encoder holds a band of the picture (EncoderQuality::band), the refresh that falls back from a constant-rate cleanup sweeps the picture a band a frame in raster order, the first FIRST_BAND of it and each next sized from the bytes of the last to BAND_BUDGETS frame budgets, so the stream keeps its rate: a whole refresh at the paint-over quantizer is 86 budgets of text at 2 Mbit/s at 1080p and queues for 240 ms on a 12 Mbit/s link.

KEY_CLEANUP_MASS 🔒

The change since the last key-frame cleanup, in screens, past which a region still for KEY_STILL_TRIGGERS trigger periods is cleaned up by a key frame.

KEY_STILL_TRIGGERS 🔒

The trigger periods a region holds still before its key-frame cleanup.

LEAST_BAND 🔒

The least share of the picture a band covers. A band that small costs next to nothing, so a held frame of one that comes out past its budget is the rest of the frame’s cost.

LEVEL_S 🔒

Seconds the finest quantizer such a session has coded the still screen at stands before its rate control counts as done with the picture, where the session says its quantizer and does not measure: at 0.25 Mbit/s at 1080p x265 takes a step finer every 2 to 8 s for a tenth of a dB a second, and libvpx’s VP9 one every 6 s after its first ten, each for all of REFINE_S at the full rate, where at 2 Mbit/s x265 takes one every 2 s for its first twenty.

LOW_MOTION_SHARE 🔒

LOW_MOTION_TRIGGERS 🔒

A region that keeps changing is still due a cleanup once this many trigger periods passed since it first changed, where at most a quarter of its frames carry more than a small change, on an average over as many frames (StripeState::motion), so the motion that went before (a scroll) ages out of it while a caret blinks.

OVERSHOOT_BUDGETS 🔒

Frame budgets the rate control’s frames of a still screen exceed for a trigger period when it is pinned at its coarsest quantizer, refining nothing within the rate: NVENC’s H.264 codes a still screen of texture at 0.1 Mbit/s at 1080p in 3.3 kB frames, 16 budgets, at quantizer 51, where at 2 Mbit/s single frames after a change reach 4.5 budgets.

PLATEAU_CHECKS 🔒

PLATEAU_GAIN_DB 🔒

PLATEAU_TRIGGERS 🔒

Trigger periods between two measurements of a picture a constant-rate cleanup is refining (FrameEncoder::last_psnr), the gain in dB under which one finds the picture level with the one before, and the level measurements in a row after which the rate control is done with it. A rate control that has a whole screen of dense text to refine at 2 Mbit/s at 1080p gains 0.2 dB a second at first, coding a few blocks of it a frame, and one that is done with a screen under 0.06.

REFINE_S 🔒

The same for a session that holds no quantizer but says the one it codes at or measures its pictures, which has no refresh to fall back to and ends its cleanup at the paint-over quantizer or once the picture stops improving: at 2 Mbit/s at 1080p x265’s rate control refines a screen of text by 0.7 dB a second, still short of that quality after ten. A band sweep runs this long at most too.

SAME_FRAME_QUALITY 🔒

SAME_FRAME_SHARE 🔒

The share of the frame before it within which a frame of a still screen codes nothing new, and the quality index from which its rate control sits at its coarsest: at 0.1 Mbit/s at 1080p libvpx’s VP9 frames are 266 bytes each at index 51, or 220-221 at 46-47, and x265’s 224 at 50-51, its strict constant rate padding them with filler, while the picture stays level, so never small against the 208-byte budget. From 1 Mbit/s their rate controls code a still screen at 36 and finer, x265’s padded frames again of one size while it refines.

SMALL_AREA 🔒

A change covering at most this fraction of the frame is small: it does not count as motion toward the low-motion cleanup.

STALL_TRIGGERS 🔒

Trigger periods of such small frames at a quantizer still coarser than the paint-over one after which the rate control counts as stalled: it stopped refining short of the paint-over quality, as NVENC’s does at 2 Mbit/s at 1080p.

SWEEP_FINER_SHARE 🔒

The share of a sweep’s frames left that the rest of the picture may take at a quantizer step finer before the sweep steps finer: room for the bands’ cost to vary down the picture.

UNKNOWN_MASS 🔒

How much of a screen a change of unknown extent counts for toward a key-frame cleanup, from the third frame of a run of changed frames on: sustained motion, where an isolated change (a caret, a clock, a keystroke) counts for nothing.

UNKNOWN_RUN 🔒

Functions

band_damage 🔒

The damage a band hash (hash_bands) reads: the fraction of the bands that changed. No bands read yet, before a capture’s first hash, is all new.

band_rects 🔒

The rows a band hash (hash_bands) found changed in a width x height frame, as full-width rectangles, one per run of changed bands: the CPU path’s dirty map. No bands read yet is the whole frame.

band_share 🔒

The share of the picture the next band of a sweep covers, after one of size whose held frame came out at bytes (FIRST_BAND onward): BAND_BUDGETS of budget, at most doubling or halving a frame.

cleanup_cap 🔒

The bytes a constant-rate cleanup’s frame may come to: the session’s own buffer, what its rate control holds a frame to, or CLEANUP_FRAME_S of the cleanup’s rate (cleanup_rate) where that is more. A still screen’s frames past what the path carries queue on it, where congestion control reads them as the path falling behind; at 60 fps and 500 kbps NVENC’s rate control refines a 1080p page a budget a frame, and now and then mispredicts one at fifty.

cleanup_due

The still-screen cleanup of one region (a whole frame, or a stripe): track how long and how much it has been changing, and say when it is due a cleanup and of which kind. keys false (JPEG, whose every stripe is a picture of its own, and a constant-rate session cleaned up through its rate control) never answers Cleanup::Key.

cleanup_frames 🔒

converge_frames for the session encoder describes (REFINE_S).

cleanup_pending

Whether cleanup_due would answer a cleanup for a frame that changed nothing, without moving the bookkeeping: what an idle frame has to know before it skips a region.

cleanup_rate

The rate (kbps) a constant-rate cleanup’s sweep is paced to and its frames capped by: what the path carries (CaptureSettings::video_link_kbps), never under the session’s rate nor over CLEANUP_RATE_MAX times it, and the session’s rate where the path’s is not known. A still screen’s refresh at the paint-over quantizer is many seconds of a low rate (1080p text at 500 kbps, 20 s), which a path faster than the stream carries at once and one near its rate only at that rate.

converge_frames

Frames a constant-rate cleanup through the rate control flows at most (CONVERGE_S).

convergence

Where the rate control stands (Convergence).

cpu_list 🔒

A kernel CPU list (“0-3,32-35”) as a set.

decide_constant_quality 🔒

decide_hw_fullframe for a constant-quality session: the cleanup moves the session’s own quality to the paint-over one, as main’s paint-over did, rather than holding a frame at a quantizer. Each encoder then codes the cleanup the way it codes any frame at that quality: x264’s and x265’s rate factor give a key frame their intra offset (about three steps finer than the paint-over index), SVT-AV1 its key-frame qindex, and the encoders whose quality change re-opens the session (x265, SVT-AV1, kvazaar) open on a key frame at it. A frame held at the paint-over quantizer itself came out coarser than main’s (SVT-AV1 49.6 against 61.0 dB, x264 53.7 against 56.5 dB).

decide_constant_rate 🔒

decide_hw_fullframe for a constant-rate session, recovery_idr the key frame a consumer asked for or the schedule’s.

decide_hw_fullframe

The send / quality / keyframe policy every full-frame encoder obeys. A constant-quality session takes decide_constant_quality; what follows is the constant-rate one.

due_cleanup 🔒

The cleanup a region is due with still frames held still, unclean frames since it first changed after its last cleanup, and motion the share of its recent frames in motion.

frame_budget

Bytes a frame of the constant-rate target, at kbps and fps.

hash_bands 🔒

Which bands of DAMAGE_BAND_ROWS rows of a host frame (stride bytes per row, height rows) changed against the frame before, read from a content hash per band (StripeState::content_dirty, one state per band in bands, with its damage blocks). The bands hash in turn on the calling thread: a band takes tens of microseconds, less than waking the rayon pool for it costs, which on a many-core host spent several times the hash in CPU and added milliseconds waiting on the slowest worker.

hash_beside 🔒

Run encode on this thread while a thread of its own hashes the frame’s bands (hash_bands), and return what encode returned with the bands that changed: a Turbo frame’s hash kept off its encode’s path. A scoped thread per frame rather than the rayon pool, whose idle workers spin after every wake; where no thread starts, the hash runs here after the encode. The thread keeps to the CPUs sharing this one’s last-level cache (share_cache_with).

hash_damage 🔒

The damage of a host frame against the one before it, from its band hash (hash_bands).

hashed_bands 🔒

The bands that changed by hashes taken where the frame was uploaded (hash_next_upload), read through each band’s state as the CPU’s own hashes are (StripeState::hash_dirty).

hashed_damage 🔒

The damage of a frame whose bands were hashed where it lies (NvFBC’s, on the GPU) against the hashes of the frame before, which last keeps: the share of the bands that changed. No hashes before is all new.

held_refresh_quality

The quality index a refresh held in one frame (a predicted frame) is coded at: the paint-over one, or, where a constant rate last coded far coarser, as much finer than that as HELD_KEY_BUDGET_S of the target allows at six steps a doubling, since a refresh of a whole screen at the paint-over quantizer after heavy motion at a low rate would cost many frames of budget at once. The bands of the sweep a cleanup through the rate control falls back to, each sized to a frame’s budget, are coded at the paint-over quantizer instead: counted from the rate control’s last quantizer, which a key frame after a loss leaves at the coarsest, the sweep ended up to 13 dB short of it at 8 fps.

last_cache_cpus 🔒

The CPUs sharing cpu’s last-level cache, the highest level its sysfs cache entries list.

paint_over_improves

Whether a cleanup improves the picture at all: at a constant quality where the paint-over quality is finer than the session’s (the cleanup then moves the session’s quality, see decide_constant_quality); at a constant rate where it is finer than the quality index the rate control last coded at, less the steps its frames need to come to the paint-over quality (EncoderQuality::finer), or the encoder does not say. A rate control that has already refined a still screen past it (a constant rate refines every frame it sends) is left alone: holding the paint-over quantizer there would coarsen the picture. A constant-rate encoder that holds no quantizer is cleaned up by the rate control’s own frames, a refresh and its burst, which it refines; a key frame there would come out at the rate control’s quantizer, which a small buffer starves.

periodic_idr_due

Whether a scheduled keyframe is due this tick.

plan_refinement

The block ledger’s part of a full-frame decision d (ledger): take this frame’s change into st’s ledger, from where the source says it changed (changed, in the frame’s coordinates, read with whether the region’s hash saw it change, StripeState::hash_changed, through BlockLedger::frame_reported), or the whole frame where the source cannot say and damage says something changed; and where the frame goes at the session’s own quality while the region moves, on a session that refines (refines, how: ledger::Refiner) with the cleanup armed, name the blocks it refines, the offset taken from the quality index the rate control last coded at (last) where a rate binds the session (bound, FrameEncoder::rate_bound), paced against the frame budget (ledger::RefinePace), or from the session’s own at a constant quality no peak binds. A capped constant quality its peak let go codes again the blocks it coded while bound that held still since, so many a frame at its own quality (BlockLedger::refresh, ledger::RefreshPace), the rest held as skips, sending a still screen’s frames for it, and first, at the paint-over quality, the blocks owed it (BlockLedger::refresh_owed): a cleanup it spent under the binding peak, or one falling due on blocks it has yet to refresh or refreshed at its own quality, which it takes in place of that cleanup’s frame; it refines nothing on those frames and waits while the peak binds again.

plateau 🔒

One frame of a cleanup that measures the picture it is refining, where the encoder can (FrameEncoder::measure): whether this frame is to be measured, one every period, and whether the measurement of the last such frame, psnr, was the PLATEAU_CHECKSth in a row to gain less than PLATEAU_GAIN_DB on the one before, the rate control having nothing left to give the picture.

recent_motion 🔒

The share of a region’s recent frames that carried more than a small change, motion taking in one more frame: an average over LOW_MOTION_TRIGGERS trigger periods.

refine_frames

Frames a constant-rate cleanup through a rate control that says its quantizer flows at most, with no refresh to fall back to (REFINE_S).

refinement_coded

What a full-frame frame decided as d left in st’s ledger once coded in bytes, the encoder now reporting quality: its refined blocks are clean where the rate control’s quality index put them at the target (BlockLedger::refinement_coded) and, where a rate binds the session (bound, as plan_refinement takes it), the pace follows the rate control (RefinePace); a frame held whole at the paint-over quality, or the region’s cleanup having run its course, cleans the region, and a key frame at the rate control’s own leaves it unclean. A frame coded under a capped constant quality’s binding peak (capped, FrameEncoder::coded_capped) leaves the blocks it coded coarse and cleans nothing; one coded without it takes the blocks it refreshed (HwFrameDecision::refresh) off the coarse ones and paces the refresh against the peak’s frame budget (ledger::RefreshPace), where it came out at all (bytes above 0). The last block owed the paint-over (BlockLedger::spent, plan_refinement) refreshed, the cleanup falls due again.

refresh_quality 🔒

held_refresh_quality from a rate control’s last frame of spent frame budgets rather than one: a rate control pinned at its coarsest quantizer (OVERSHOOT_BUDGETS) refreshed in one held frame, which then costs that much more.

refresh_start

The still frames a refresh after the region moved waits for (cleanup_due’s start): a refresh through the rate control (converges) is that rate control’s own frames, each within its budget, so it starts on the first still frame, where waiting trigger frames only left the picture as the motion did; a refresh held at the paint-over quantizer waits for the screen to stay still. A region that changed only a little (a caret, a keystroke) waits trigger either way, or a caret blinking on a clean screen would send every frame.

share_cache_with 🔒

Keep this thread to the CPUs it may run on that share cpu’s last-level cache.

start_sweep 🔒

Start a refresh sweep at the top of the picture: its first band in this frame at qp, the rest a band a frame after it (FIRST_BAND, band_share).

sweep_refresh 🔒

A refresh due on a session that sweeps it (decide_hw_fullframe’s sweeps): a sweep from the top, or, with one under way, which is already refreshing the picture, nothing held, the frame going at the rate control’s quality: a sweep started over at every change of a blinking caret would never cover the picture, and a whole held frame is what the sweep spreads.