trikkeSensors/docs/binary-record-v1.md

TRK1 Binary Telemetry — Version 1

The binary stream is the shared transport and storage representation for USB, BLE, and any later nonvolatile buffer. All multibyte integers and IEEE-754 float32 values are little-endian. Frames are self-identifying and may be split or combined by an underlying byte transport.

Frame header (36 bytes)

Offset	Size	Field
0	4	ASCII magic TRK1
4	1	Wire version (1)
5	1	Packet type: metadata 1, samples 2, status 3
6	1	Header size (36)
7	1	Record size (0 or 20)
8	1	Record count (0 or 1–8)
9	1	Packet flags
10	2	Payload size
12	4	Monotonic packet sequence
16	8	Base ESP timer timestamp in microseconds
24	4	Cumulative samples lost to sensor read failure or queue overflow
28	4	Cumulative acquisition-loop overruns
32	4	IEEE CRC-32

CRC uses polynomial 0xEDB88320, initial value 0xFFFFFFFF, and final XOR 0xFFFFFFFF. It covers header bytes 4–31 followed by the complete payload. The magic and stored CRC field are excluded.

Packet flag bit 0 means at least one sample timestamp delta saturated. Both host tools report any frame carrying this flag instead of silently treating its reconstructed timestamps as exact.

Sample record (20 bytes)

Offset	Size	Field
0	4	Sample sequence
4	2	Timestamp delta from the preceding record, in 10 us units
6	2	Enclosure accel X raw (int16)
8	2	Enclosure accel Y raw (int16)
10	2	Enclosure accel Z raw (int16)
12	2	Enclosure gyro X raw (int16)
14	2	Enclosure gyro Y raw (int16)
16	2	Enclosure gyro Z raw (int16)
18	1	Raw ADXL345 INT_SOURCE
19	1	Raw L3G4200D STATUS_REG

The first record has delta zero and uses the frame's base timestamp. Each later timestamp is reconstructed by cumulatively adding its delta. Firmware ends the current packet before a delta exceeds the representable 655.35 ms range, making the next sample the exact base timestamp of a new packet. As a defensive encoder fallback, an unrepresentable delta is stored as 0xFFFF and sets packet flag bit 0. Because intra-packet deltas are rounded to 10 us while each packet base keeps the exact ESP timer value, an integrity check for an exact 10,000 us interval can report up to +/-5 us at packet boundaries; changing the transport's packet size changes how often that harmless quantization boundary appears. Sample sequence gaps remain detectable independently.

Mapped raw counts are authoritative. The original sensor-native axes can be reconstructed because the mappings are lossless:

accel native = (-enclosure_y, enclosure_x, enclosure_z)
gyro native  = ( enclosure_x, enclosure_y, enclosure_z)

Metadata payload (48 bytes)

Metadata frames repeat approximately every five seconds so a receiver may attach midstream. The payload contains:

• Sample rate, accelerometer range, gyroscope range, and mapping/calibration flags
• Three accel offset float32 values
• Three accel counts/g float32 values
• Three gyro bias float32 values
• Nominal gyro mdps/LSB float32 value

A byte-stream receiver may begin inside an incomplete frame. It discards bytes until a magic/header/CRC combination validates. Host tools report any rejection before that first valid frame separately from CRC failures after synchronization.

Status payload (32 bytes)

Status frames use packet type 3, record size/count zero, and payload version 1. They are emitted at startup and approximately every five seconds. Offsets 0 and 2 are uint16 payload version and payload size; the remaining fields are cumulative uint32 counters:

Offset	Field
4	Sensor read failures
8	Sample-queue overflows
12	Initial transport submissions that accepted zero bytes
16	BLE disconnects after a connection was established
20	BLE notification enqueue/send failures
24	BLE frame replays after disconnect, subscription change, or ACK timeout
28	Malformed, stale, premature, or wrong-connection ACK writes

The header's cumulative dropped-sample count remains the sum of sensor read failures and queue overflows, preserving version-1 receiver compatibility while the status payload makes the causes independently observable. USB-specific BLE counters remain zero. The rejected-ACK total can also include a harmless valid duplicate that arrives after its frame has already completed, so a nonzero value does not by itself prove corruption or a hostile receiver.

Buffering

Acquisition runs in a dedicated higher-priority task and writes complete samples to a 1024-entry RAM queue. The lower-priority output task batches up to eight records per frame. At 100 Hz this queue represents about 10.24 seconds of decoupling when the transport reports backpressure or failure accurately. A failed write retains and retries the same encoded packet rather than dequeuing more samples, so the queue accumulates the outage backlog. After reconnection, the oldest retained data is sent first. If the queue fills, acquisition drops new samples rather than overwriting older ones; sequence gaps and the cumulative lost-sample counter expose that permanent loss.

The shared transport state machine distinguishes three nonfatal states. RETRY is valid only from initial submission: it means zero bytes were accepted and the complete frame may be submitted again. PENDING means the backend owns an in-flight frame, so firmware may only poll that transfer. A RETRY returned by polling fails closed as FATAL, because generic code cannot prove whole-frame resubmission is duplicate-safe. COMPLETE permits the output task to reuse its packet buffer and consume more samples. This prevents a timeout after partial progress from causing an ambiguous whole-frame duplicate.

The direct USB Serial/JTAG backend atomically copies a complete frame into its TX ring, then polls a bounded transmit-drain wait. A timeout remains PENDING; it does not trigger resubmission. This closes the VFS/stdio path's silent-discard case for a connected host that stops draining.

USB drain is not end-to-end application delivery confirmation. A host process may attach after earlier frames have already left the endpoint, or fail after the endpoint accepts them. CRC and sequence checks make resulting loss visible, but an application acknowledgement and replay window are still required to guarantee receipt. Accordingly, USB COMPLETE means endpoint drain, while reliable BLE reserves COMPLETE for an application ACK of the exact frame. See ble-transport-v1.md for fragmentation, replay, and UUIDs.

Receivers report bytes left in an incomplete trailing frame when capture ends. Those bytes cannot pass CRC validation and are not silently admitted as samples.