add framed binary telemetry transport

README.md

@@ -7,9 +7,9 @@ This milestone does four things:
 
 1. Detects and verifies both sensors by their identification registers.
 2. Configures each sensor for a nominal 100 Hz raw output rate.
-3. Emits timestamped, sensor-native raw readings over the XIAO USB connection.
-4. Maps both sensors into a shared enclosure coordinate frame and emits both raw
-   and calibrated readings.
+3. Emits framed, timestamped binary readings over the XIAO USB connection.
+4. Maps both sensors into a shared enclosure frame and carries the metadata
+   needed to derive calibrated readings without replacing raw data.
 
 BLE transport and phone-side storage come after the wired sensor path is proven.
 
@@ -49,17 +49,27 @@ enclosure Y = -native X
 enclosure Z =  native Z
 ```
 
-Software calibration is applied after enclosure-axis mapping. Accelerometer
-offset and per-axis scale were measured with a six-face enclosure test. Gyroscope
-zero-rate bias and polarity were measured; its 17.5 mdps/LSB scale remains the
-nominal datasheet value. Sensor-native and mapped raw counts remain in every
-record for diagnostics. No software filtering or sensor fusion is performed yet.
+Host-side software calibration is applied after enclosure-axis mapping.
+Accelerometer offset and per-axis scale were measured with a six-face enclosure
+test. Gyroscope zero-rate bias and polarity were measured; its 17.5 mdps/LSB scale
+remains the nominal datasheet value. Mapped raw counts are stored directly, and
+sensor-native counts are reconstructed losslessly from the documented mapping.
+No software filtering or sensor fusion is performed yet.
 
-The ESP32-C3 polls at exactly 100 Hz, but each sensor has an independent internal
+The ESP32-C3 polls at exactly 100 Hz in a dedicated acquisition task, but each
+sensor has an independent internal
 sample clock. The status registers are read immediately before each XYZ read so a
 consumer can distinguish a fresh sample from a repeated poll and identify gyro
 overruns. Hardware data-ready interrupts and FIFO acquisition are deferred to the
-buffering milestone.
+later sensor-side acquisition refinement.
+
+Completed samples enter a 128-record RAM queue. A lower-priority output task
+batches up to eight records into versioned `TRK1` frames, isolating acquisition
+from brief USB or future BLE stalls. CRC, packet and sample sequences, timestamps,
+and cumulative loss/overrun counters make loss detectable.
+
+Measured end-to-end framing overhead is about 2.47 kB/s at 100 Hz, or 8.47
+MiB/hour before BLE link overhead.
 
 ## Build and flash
 
@@ -74,20 +84,24 @@ For each new terminal:
 ```sh
 source /Users/jay/.espressif/v6.0.2/esp-idf/export.sh
 idf.py build
-idf.py -p /dev/cu.usbmodem1134101 flash monitor
+idf.py -p /dev/cu.usbmodem1134101 flash
 ```
 
-Exit the serial monitor with `Ctrl-]`.
-
 ## USB output
 
-After startup metadata, records use CSV:
+After readable startup metadata, the device emits framed binary. Each sample is a
+20-byte record containing mapped raw sensor counts, timing, sequence, and the two
+raw status bytes. See [the complete wire-format specification](docs/binary-record-v1.md).
+
+Binary is the authoritative capture format. The host tools render it back to the
+same diagnostic CSV schema used during calibration:
 
 ```text
 sequence,poll_timestamp_us,accel_x_raw,accel_y_raw,accel_z_raw,gyro_x_raw,gyro_y_raw,gyro_z_raw,accel_x_mg,accel_y_mg,accel_z_mg,gyro_x_mdps,gyro_y_mdps,gyro_z_mdps,accel_native_x_raw,accel_native_y_raw,accel_native_z_raw,gyro_native_x_raw,gyro_native_y_raw,gyro_native_z_raw,accel_int_source,gyro_status,loop_overrun_count
 ```
 
-`poll_timestamp_us` is the ESP32-C3 monotonic time immediately before the status
+`poll_timestamp_us` is reconstructed from each frame's base timestamp and 10 us
+record deltas. It represents the ESP32-C3 monotonic time immediately before status
 and data reads. It is not the sensors' physical sample time. The axes in the first
 six sample columns use the enclosure frame above. Calibrated acceleration is in
 integer milligravity (`mg`), and bias-corrected angular rate is in integer
@@ -104,9 +118,19 @@ Status bits:
 - `loop_overrun_count`: cumulative acquisition deadlines missed; the loop
   resynchronizes after a miss instead of issuing catch-up bursts.
 
-The capture tool auto-detects a single `/dev/cu.usbmodem*` device, writes only
-validated numeric records to a real CSV, and reports sequence or timing problems:
+The binary capture tool auto-detects a single `/dev/cu.usbmodem*` device, stores
+only CRC-valid frames, renders CSV, and reports packet, sample, timing, status,
+drop, and overrun totals:
 
 ```sh
-python tools/capture_serial.py
+python tools/capture_binary.py
 ```
+
+An existing `.trk` stream can be decoded again without hardware:
+
+```sh
+python tools/decode_binary.py captures/session.trk captures/session.csv
+```
+
+`tools/capture_serial.py` remains available only for decoding captures from the
+older CSV-v3 firmware snapshots.