retroDE_ps2/docs/decisions/0011-gs-dump-ingestion.md

# 0011 — GS dump ingestion (Ch340): parse, census, translate a supported subset

Status: **ACCEPTED — Ch340 CLOSED (2026-06-21)** as a parser + census / fail-closed victory. Brick 5
(authentic on-silicon render) is **explicitly waived** because the authentic dump contains zero
supported segments — that is the census doing its job, not a failure.

## Closeout (honest framing, per Codex)
- Authentic `cubes_demo` GS dump (MIT homebrew, content-clean) parsed **deterministically, 0 malformed**.
- Container format pinned from PCSX2 source and validated byte-exact; byte-exact synthetic parser test
  passes (`tools/test_gs.sh`).
- Primitive reconstruction (GS vertex-kick model) works: **648 triangles + 540 sprites**.
- Support census classified **every** primitive; histograms + reasons emitted to
  `captures/gs/reports/cubes.census.txt` (aggregate only).
- Translator **failed closed with no scene**: every authentic triangle is textured (`TME=1`, no real-
  texture path) and sprites are unsupported. Nothing was approximated.
- Core trust-boundary goal achieved: **authentic GS traffic enters the pipeline and unsupported
  content is reported, not faked.** The translator→`ps2_feeder`→staging path is proven on the
  supported synthetic fixture; authentic silicon render is deferred to the census-derived blocker.
- Mechanical top blocker → **Ch341: textured-triangle ingestion** (real texture state/upload/bind).
  Do NOT hunt another dump for a convenient flat segment; do NOT substitute synthetic silicon for
  authentic Brick 5.

Original design follows (the boundary it set still holds).

## Goal & trust boundary
Authentic GS traffic enters the proven host pipeline, is decoded **deterministically**, and a
**strictly-supported subset** reaches pixels with **no hidden approximation**. Ch340's honest victory
is that property — NOT a full game frame rendering. Real captures will expose unsupported textures,
transfers, blend/state, and primitive modes; those are **reported**, never approximated.

## Pipeline
```
.gs[.xz]  ──(container parser)──►  raw packets  ──(GIF/GS decoder)──►  normalized event stream
   │                                                                          │
   │ (local, gitignored)                                                      ▼
   └────────────────────────────────────────────────►  support census + histograms (reports/)
                                                                              │
                                              supported subset ──(translator)─┴─►  ps2_feeder scene file
                                                                                    (Ch339 encoder streams it)
```
The translator emits the Ch339 text scene grammar (`tri`/`trig`/`tritile`/`rect`/`go`); it does NOT
re-implement the staging format. `ps2_feeder -f scene.txt` renders it on the existing bitstream.

## Normalized event stream (schema v1, versioned)
A flat, ordered list; every event carries `byte_off`, `frame_idx`, `event_idx`. Event kinds:
- `FRAME_BOUNDARY {field}` — VSync packet (frame delimiter).
- `GIFTAG {path, nloop, eop, pre, prim, flg, nreg, regs}` — decoded GIF tag header.
- `GSREG {addr, name, value}` — a GS register write (from A+D, REGLIST, or PACKED expansion).
- `IMAGE {qwc, dst_fmt}` — an IMAGE-mode (HWREG/texture/FB upload) transfer; payload bytes summarized,
  NOT inlined into committable output.
- `READFIFO {qwc}` — local→host transfer.
- `MALFORMED {reason}` — structural decode failure at this offset.

`GSREG.name` covers the register set we already encode in `bake.py`/`ps2_feeder`: PRIM, RGBAQ, ST, UV,
XYZ2/XYZ3, XYZF2/XYZF3, TEX0_1/2, CLAMP, FOG, TEX1/2, FRAME_1/2, ZBUF_1/2, TEST_1/2, ALPHA_1/2,
SCISSOR, PRMODE/PRMODECONT, BITBLTBUF, TRXPOS, TRXREG, TRXDIR, etc. Unknown addrs → `GSREG` with
`name="UNKNOWN_0xNN"` (reported, not dropped).

## Support census (every event classified)
- **translated** — emitted into the ps2_feeder scene (the supported subset, below).
- **ignored (justified)** — safely skippable with a stated reason (e.g. FOG with FGE off; a redundant
  state write; a NOP). The justification is explicit per category.
- **unsupported** — a real effect we cannot faithfully reproduce yet (textured prim, sprite with a
  real texture, blend mode ≠ the proven source-over, PSM we don't render, Z format, dest-alpha test,
  scissor we don't honor, TRIANGLE_FAN/STRIP we haven't reduced, lines/points, IMAGE texture upload).
  Recorded with frame/event/byte offset + the exact reason. **Never approximated.**
- **malformed** — structural failure (bad GIF tag, truncated payload, length mismatch).

Reports (committable, no game content): per-dump JSON/text with frame count, a GS-register-write
histogram, a primitive-mode histogram, and the full unsupported/malformed list with offsets+reasons.

## Supported subset that reaches pixels (Ch340 v1)
Matches what the feeder + `ps2_feeder` already render faithfully on silicon:
- `PRIM` = TRIANGLE (prim type 3), with `IIP` flat or gouraud (per-vertex RGBAQ).
- Vertices via `XYZ2`/`XYZ3` (Z honored — Ch338 cross-batch Z is correct).
- Color via `RGBAQ` (MODULATE through the unity texel — matches the proven path).
- `FRAME_1`/`ZBUF_1`/`TEST_1`(GEQUAL)/`ALPHA_1`(the proven source-over) within the supported envelope.
A draw segment qualifies only if EVERY primitive in it is in this subset and the state matches the
proven envelope. Sprites→rect and triangle-strip→triangle reduction are candidate Ch341 work, decided
from the census, not pre-built.

## Acceptance (Codex)
1. Byte-exact parser tests on a small **synthetic** `.gs` fixture (`captures/gs/synthetic/`, authored
   once the real container format is confirmed).
2. One authentic dump parses **deterministically** (same events every run).
3. Frame / register / primitive histograms emitted.
4. Unsupported events carry frame/event/byte offset + reason.
5. ≥1 carefully chosen supported frame/segment translates to a ps2_feeder scene and **renders on
   silicon**.
6. Translation failures **stop before board access**.

## Bricks (gated on the dump)
1. Inspect the real dump bytes; confirm the container framing (header, compression, packet types).
   Build the container parser + a matching synthetic fixture; byte-exact parser test.
2. GIF/GS decoder → normalized events (GIF tag + A+D/REGLIST/PACKED register expansion). Unit-tested
   against hand-built GIF packets (the encode side already exists in `bake.py`).
3. Census + histograms over the normalized stream; emit reports.
4. Translator: supported-subset events → ps2_feeder scene file; everything else → census. Fail closed.
5. Pick a supported segment from the census; render via `ps2_feeder -f`; confirm on silicon.

Ch341 is then chosen from the census's highest-impact blocker, not guessed now.