83 lines
3.7 KiB
Markdown
83 lines
3.7 KiB
Markdown
# Binary Transport Validation — 2026-08-17
|
|
|
|
This milestone replaced high-volume device-side CSV with the versioned `TRK1`
|
|
binary stream shared by future USB, BLE, and storage paths. Mapped raw sensor
|
|
counts remain authoritative. Calibration metadata travels as float32 values, and
|
|
the host reconstructs the prior 23-column diagnostic CSV without discarding raw
|
|
data.
|
|
|
|
## Verification layers
|
|
|
|
- Host compilation of the production C encoder with `-Wall -Wextra -Werror`
|
|
- Fragmented C-encoder-to-Python-parser contract test
|
|
- Deliberately corrupted CRC test with stream resynchronization
|
|
- Full eight-record, timestamp-saturation, invalid-size/count, and truncated-tail
|
|
encoder/parser contract cases
|
|
- ESP-IDF firmware build and flash on the assembled ESP32-C3 prototype
|
|
- Live USB capture followed by independent offline re-decoding
|
|
|
|
## USB text-conversion finding
|
|
|
|
The first live capture exposed that the USB console's default CRLF mode inserted
|
|
a carriage return whenever a binary byte equaled LF (`0x0A`). CRC rejected every
|
|
affected frame and the parser resynchronized at the next `TRK1` magic. No invalid
|
|
sample entered decoded CSV.
|
|
|
|
Before binary output begins, firmware now changes the USB Serial/JTAG VFS transmit
|
|
mode to `ESP_LINE_ENDINGS_LF`, which means no byte modification. Startup logs and
|
|
readable metadata are flushed first.
|
|
|
|
## Final hardware capture
|
|
|
|
`captures/binary_v1_smoke2.trk` and its decoded CSV contain:
|
|
|
|
- 7,184 samples over 71.830 seconds
|
|
- 912 total frames, including 14 repeated metadata frames
|
|
- Packet gaps and resets: 0
|
|
- Sample gaps and resets: 0
|
|
- Timestamp anomalies: 0
|
|
- CRC and header failures: 0
|
|
- Queue/read/output drops: 0
|
|
- Acquisition-loop overruns: 0
|
|
- ADXL345 overruns: 0
|
|
- L3G4200D data-ready clear: 0
|
|
- L3G4200D overruns: 192
|
|
|
|
Offline decoding of the saved `.trk` file produced CSV byte-for-byte identical to
|
|
the CSV rendered during live capture.
|
|
|
|
The validated stream occupied 177,184 bytes, or 2,466.7 bytes/s including frame
|
|
headers and repeated metadata. That is 8.47 MiB/hour and about 19.7 kbit/s before
|
|
BLE link overhead, far below the previous CSV stream.
|
|
|
|
## Task separation and buffer
|
|
|
|
The 100 Hz I2C acquisition runs at FreeRTOS priority 10. USB encoding/output runs
|
|
at priority 5 and receives samples through a 512-entry queue (about 5.12 seconds
|
|
at 100 Hz). The hardware capture's zero timing anomalies and zero loop overruns
|
|
confirm that packet encoding, CRC, float metadata, and USB output did not disturb
|
|
the acquisition cadence.
|
|
|
|
Output failure is transactional: firmware retains and retries the same encoded
|
|
packet with a scheduler delay instead of discarding it or dequeuing more samples.
|
|
The queue therefore accumulates a disconnected-transport backlog. If an outage
|
|
outlasts the queue, acquisition drops and counts new samples while preserving the
|
|
oldest queued data for ordered delivery after reconnection.
|
|
|
|
## Forced transport-outage validation
|
|
|
|
A temporary validation build made the packet writer report transport failure for
|
|
a fixed interval while acquisition continued normally. The failure injection was
|
|
removed before the production build.
|
|
|
|
With a three-second forced outage, all 2,144 observed samples arrived contiguously
|
|
from sequence 0 through 2,143. Packet gaps, sample gaps, drops, loop overruns, and
|
|
timestamp-saturation flags were all zero.
|
|
|
|
With a seven-second forced outage, the queue preserved samples 0 through 511 and
|
|
then dropped 138 new samples after reaching capacity. Delivery resumed at sample
|
|
650. The cumulative drop count and observed sequence gap both equaled 138. The
|
|
timestamp difference from sample 511 to 650 was exactly 1,390,000 us, matching
|
|
139 sample intervals, and no saturation flag was emitted. This verifies both the
|
|
oldest-data retention policy and the new exact timestamp re-anchor after overflow.
|