Frame

A read/write view of a raw captured frame, passed to OnFrameReceived. Byte access is 0-based into the pixel buffer, whose layout depends on format() — capture frames are typically NV12 (a full-size Y/luma plane, then an interleaved half-size UV plane).

methodframe:width() → number

Frame width in pixels. Takes no parameters.

methodframe:height() → number

Frame height in pixels. Takes no parameters.

methodframe:pitch() → number

Bytes per row of the first (luma) plane. Index a pixel's row with row * pitch() + col. Takes no parameters.

methodframe:format() → string

The pixel format name, e.g. "SDL_PIXELFORMAT_NV12". Takes no parameters.

methodframe:size() → number

Total pixel-buffer size in bytes — the valid range for byte indices. Takes no parameters.

methodframe:getByte(i) → number

Reads one byte of the pixel buffer.

Param	Type	Notes
i	number	0-based byte index (0 .. `size()-1`). Out of range returns 0.

methodframe:region(x, y, w, h) → string

Read a rectangle in one call. Returns w*h bytes, row-major — index it with string.byte. Anything outside the buffer comes back as 0 rather than being refused.

Param	Type	Notes
x, y	number	Top-left, in bytes across and rows down the pixel buffer.
w	number	Width in bytes, capped at 4096.
h	number	Rows, capped at 8192 — enough for the extra chroma rows below.

Rows run over the whole buffer, not just the picture. On a planar format the interleaved chroma plane sits immediately after the luma rows, so a 1920×1080 NV12 frame is 1620 readable rows, not 1080 — frame:size() is the byte count that covers, and this is the only way to read it in bulk. That is what makes a whole frame reachable: frame:region(0, 0, w, h + math.ceil(h/2)) hands back one packed NV12 frame, padding excluded, which is exactly what a recorder or an exporter needs.

Use this, not a loop of getByte. Every call from Lua into the engine costs about 1.6µs whatever it carries, so the price of reading an area is its pixel count: one 150×38 window was 5700 calls and about 9ms — most of a frame at 60fps. The same window read a row at a time is 38 calls, and the per-pixel work stays in Lua where it's cheap. Reading the Pokémon mod's windows this way took it from 51fps back to 60 while scanning four times as often.

-- one row of the luma plane, sampled every 6th pixel
local row = frame:region(x0, y, 600, 1)
for i = 1, 600, 6 do
    local luma = row:byte(i)
end

methodframe:setByte(i, v) → nil

Writes one byte of the pixel buffer — this is how you change what's displayed.

Param	Type	Notes
i	number	0-based byte index. Out of range is ignored.
v	number	Byte value, 0–255.

methodframe:into(buf, at, x, y, w, h [, step, unit]) → number

The same rectangle as region, written into a Buffer you already own instead of into a fresh Lua string. Returns how many bytes landed. Rows are written back to back at w bytes each, so what arrives is packed with any row padding left behind.

Param	Type	Notes
buf	Buffer	Destination. Anything that would land past its end is dropped, not wrapped.
at	number	0-based offset in `buf` to start writing at.
x, y, w, h	number	As `region` — except that with `step`, `w` and `h` describe what is written, not what is read.
step	number	Optional, default 1. Take every `step`-th row and every `step`-th sample across: 2 is half size, 4 is quarter. Nearest sample, not an average.
unit	number	Optional, default 1. Bytes per sample. NV12's chroma plane is interleaved U,V pairs, so halving it across means keeping every other pair — pass `unit = 2` for that plane, or the colour comes out sampled off by one and smears.

This is what makes keeping frames possible. A string is immutable, so a buffer of frames is one allocation and one dead object per frame — hundreds of megabytes a second at 1080p, given back whenever the collector decides. Written into a Buffer, a ring is a cursor over bytes overwritten in place: nothing allocated, nothing freed. The same goes for step: shrinking a frame in Lua means a pass over three megabytes, which measured 128ms a frame — eight frames a second — where this is a strided copy costing a couple of ms.

-- a whole NV12 frame, packed, into a slot of your own storage
local rows = h + math.ceil(h / 2)
frame:into(buf, slot, 0, 0, w, rows)

-- the same frame at half size: luma one byte a sample, chroma in U,V pairs
frame:into(buf, slot,            0, 0, w//2, h//2,          2, 1)
frame:into(buf, slot + w//2*h//2, 0, h, w//2, math.ceil(h//2/2), 2, 2)

One page of the MCCP modding reference. Every section is its own document; search covers all of them.