add framed binary telemetry transport

2026-08-17 11:13:53 -04:00
parent a5c3087ee4
commit aceaa2b270
13 changed files with 1320 additions and 110 deletions
@@ -0,0 +1,57 @@
+# Binary Transport Validation — 2026-08-17
+
+This milestone replaced high-volume device-side CSV with the versioned `TRK1`
+binary stream shared by future USB, BLE, and storage paths. Mapped raw sensor
+counts remain authoritative. Calibration metadata travels as float32 values, and
+the host reconstructs the prior 23-column diagnostic CSV without discarding raw
+data.
+
+## Verification layers
+
+- Host compilation of the production C encoder with `-Wall -Wextra -Werror`
+- Fragmented C-encoder-to-Python-parser contract test
+- Deliberately corrupted CRC test with stream resynchronization
+- ESP-IDF firmware build and flash on the assembled ESP32-C3 prototype
+- Live USB capture followed by independent offline re-decoding
+
+## USB text-conversion finding
+
+The first live capture exposed that the USB console's default CRLF mode inserted
+a carriage return whenever a binary byte equaled LF (`0x0A`). CRC rejected every
+affected frame and the parser resynchronized at the next `TRK1` magic. No invalid
+sample entered decoded CSV.
+
+Before binary output begins, firmware now changes the USB Serial/JTAG VFS transmit
+mode to `ESP_LINE_ENDINGS_LF`, which means no byte modification. Startup logs and
+readable metadata are flushed first.
+
+## Final hardware capture
+
+`captures/binary_v1_smoke2.trk` and its decoded CSV contain:
+
+- 7,184 samples over 71.830 seconds
+- 912 total frames, including 14 repeated metadata frames
+- Packet gaps and resets: 0
+- Sample gaps and resets: 0
+- Timestamp anomalies: 0
+- CRC and header failures: 0
+- Queue/read/output drops: 0
+- Acquisition-loop overruns: 0
+- ADXL345 overruns: 0
+- L3G4200D data-ready clear: 0
+- L3G4200D overruns: 192
+
+Offline decoding of the saved `.trk` file produced CSV byte-for-byte identical to
+the CSV rendered during live capture.
+
+The validated stream occupied 177,184 bytes, or 2,466.7 bytes/s including frame
+headers and repeated metadata. That is 8.47 MiB/hour and about 19.7 kbit/s before
+BLE link overhead, far below the previous CSV stream.
+
+## Task separation and buffer
+
+The 100 Hz I2C acquisition runs at FreeRTOS priority 10. USB encoding/output runs
+at priority 5 and receives samples through a 128-entry queue (about 1.28 seconds
+at 100 Hz). The hardware capture's zero timing anomalies and zero loop overruns
+confirm that packet encoding, CRC, float metadata, and USB output did not disturb
+the acquisition cadence.