trikkeSensors/docs/transport-layer-validation-2026-08-17.md

Direct USB Transport Validation — 2026-08-17

This milestone moved framed telemetry off stdio and the USB Serial/JTAG VFS data path. The output task now targets a transport-neutral state machine backed by ESP-IDF's interrupt-driven USB Serial/JTAG driver.

Transaction contract

• RETRY: valid only from initial submission; the backend accepted zero bytes, so whole-frame resubmission is safe.
• PENDING: the backend owns an in-flight frame. Only completion polling is allowed; the caller must retain and not modify the packet buffer.
• COMPLETE: the backend's documented completion criterion is satisfied and the caller may reuse the packet buffer. USB means endpoint drain; reliable BLE will mean application acknowledgement of the exact frame.
• FATAL: a programming or backend invariant failed. The output task stops consuming the sample queue rather than silently discarding its in-flight data. RETRY from a pending poll is promoted to FATAL because generic code cannot prove resubmission is duplicate-safe.

For USB, submission uses a 512-byte TX ring and a bounded 50 ms write. ESP-IDF's ring-buffer send is all-or-nothing for each TRK1 frame. Once accepted, bounded 50 ms usb_serial_jtag_wait_tx_done() calls continue returning PENDING until the host drains the endpoint. The first completion poll happens immediately after acceptance; repeated pending polls are scheduler-paced. A timeout never resubmits the frame.

The VFS is switched to driver mode after readable startup output so any unexpected diagnostic output cannot race the driver's ISR by accessing the hardware FIFO directly. Firmware logs are disabled before binary telemetry tasks start, as before. A terminal transport invariant re-enables error logging, emits one final diagnostic, and suspends the output task. Because no later binary frame can follow, that diagnostic cannot corrupt a recoverable stream.

Queue-allocation and task-creation failure paths switch the VFS back to its non-driver mode and uninstall the USB driver before emitting their error, so the only startup diagnostic is not stranded in a TX ring that is immediately freed. The USB backend's zero-write-to-RETRY mapping depends explicitly on its checked initialized/sole-owner lifecycle; a torn-down backend fails the guard as FATAL instead of masquerading as backpressure.

Verification

The host C transport fixture compiles the production state machine with -Wall -Wextra -Werror and verifies:

• zero-accept submissions remain retryable;
• accepted transfers become pending;
• repeated pending polls never call submission again;
• completion returns the sender to idle;
• retry returned after acceptance fails closed without resubmission;
• invalid arguments and unknown backend states fail closed.

The assembled ESP32-C3 prototype first produced a normal 1,728-sample pre-commit direct-driver capture with no packet gaps, sample gaps, resets, CRC failures, reported drops, loop overruns, trailing partial bytes, or timestamp-saturation frames.

After commit 3c95f3d was built and flashed exactly, a second capture contained 1,680 contiguous samples, sequences 0 through 1,679, with the same zero-loss integrity result. Its optional raw wire file contains all 563 startup bytes and re-decodes to CSV byte-for-byte identical to the live-rendered CSV. Both sides of that exact capture are tracked as fixtures:

• tests/fixtures/direct_usb_3c95f3d.trk, SHA-256 f495486f094a758bb785e145026e3934d60b52dd5083aef7f5d896195d967869
• tests/fixtures/direct_usb_3c95f3d.wire, SHA-256 3bdaeadff7962c6eac48c4ebeda285c8eb359e439d5e1728104add2009122c03

For the connected-stall case, the USB endpoint was left enumerated without a serial reader long enough to overflow the 512-sample acquisition queue. When the reader opened, delivery preserved the oldest block through sequence 511 and resumed at sequence 1,706. The one observed gap and the cumulative device drop count both equal 1,194 samples. The timestamp difference across that gap is 11,950,000 us, exactly 1,195 sample intervals. Packet gaps, CRC failures, loop overruns, trailing partial bytes, and timestamp-saturation flags are zero.

That exact validated stream is tracked as tests/fixtures/direct_usb_stall.trk, SHA-256 40f874b7eaa7f705524ecdd75f832e8a724252366633116ac015fc75dfd16558, and its signature is asserted by the regression suite.

Remaining delivery boundary

The stall capture begins at sample sequence 8. The flashing process still had the serial endpoint open long enough to drain sequences 0 through 7 before the capture application attached. This is not silent device-side loss; it precisely demonstrates the boundary of USB drain confirmation. Only an application-level ACK can prove that the intended receiver received and persisted a frame.

The planned BLE backend will use the same PENDING ownership rule while waiting for acknowledgements, retain unacknowledged frames for replay, and expose per-cause transport/queue counters separately.

Host evidence handling

Live capture and offline decode now share one integrity tracker for packet and sample gaps/resets, timestamp anomalies, sensor status, saturation flags, drops, and acquisition overruns. Sequence classification is wrap-aware. The capture tool also accepts --wire PATH to preserve every serial byte before parsing, including startup text, corrupt frames, and trailing fragments.

For deterministic fresh-session validation, capture_binary.py --reset releases DTR/RTS, clears the prior input session, then pulses the C3 reset line while the same reader remains open. This follows ESP-IDF monitor's USB Serial/JTAG reset ordering and avoids a flash-to-capture port-open race.

Commit 73e5680 was built, flashed, and then captured through this reset path. The result contains 1,184 contiguous samples, sequences 0 through 1,183, with zero packet/sample gaps, resets, CRC failures, reported drops, loop overruns, trailing bytes, or timestamp saturation. Raw-wire offline decoding produced CSV byte-for-byte identical to live rendering. Both artifacts are tracked:

• tests/fixtures/direct_usb_73e5680.trk, SHA-256 fd34bb3bf8f92a64960024f1287e553c03076ff3714fe629ec629bec81ddf821
• tests/fixtures/direct_usb_73e5680.wire, SHA-256 82d6d17bbf0729e9bfc53f337ec9adf70bcb5e5898b039685eda5dfa19cf4eea