retain telemetry across transport outages

This commit is contained in:
Jay
2026-08-17 14:00:14 -04:00
parent bc4a2856e1
commit 291c6b54e6
11 changed files with 222 additions and 37 deletions
+18 -7
View File
@@ -19,7 +19,7 @@ combined by an underlying byte transport.
| 10 | 2 | Payload size |
| 12 | 4 | Monotonic packet sequence |
| 16 | 8 | Base ESP timer timestamp in microseconds |
| 24 | 4 | Cumulative samples lost to read failure, queue overflow, or output failure |
| 24 | 4 | Cumulative samples lost to sensor read failure or queue overflow |
| 28 | 4 | Cumulative acquisition-loop overruns |
| 32 | 4 | IEEE CRC-32 |
@@ -27,7 +27,9 @@ CRC uses polynomial `0xEDB88320`, initial value `0xFFFFFFFF`, and final XOR
`0xFFFFFFFF`. It covers header bytes 431 followed by the complete payload. The
magic and stored CRC field are excluded.
Packet flag bit 0 means at least one sample timestamp delta saturated.
Packet flag bit 0 means at least one sample timestamp delta saturated. Both host
tools report any frame carrying this flag instead of silently treating its
reconstructed timestamps as exact.
## Sample record (20 bytes)
@@ -45,9 +47,11 @@ Packet flag bit 0 means at least one sample timestamp delta saturated.
| 19 | 1 | Raw L3G4200D `STATUS_REG` |
The first record has delta zero and uses the frame's base timestamp. Each later
timestamp is reconstructed by cumulatively adding its delta. A delta that cannot
fit is stored as `0xFFFF` and sets packet flag bit 0. Sample sequence gaps remain
detectable independently.
timestamp is reconstructed by cumulatively adding its delta. Firmware ends the
current packet before a delta exceeds the representable 655.35 ms range, making
the next sample the exact base timestamp of a new packet. As a defensive encoder
fallback, an unrepresentable delta is stored as `0xFFFF` and sets packet flag bit
0. Sample sequence gaps remain detectable independently.
Mapped raw counts are authoritative. The original sensor-native axes can be
reconstructed because the mappings are lossless:
@@ -77,5 +81,12 @@ before that first valid frame separately from CRC failures after synchronization
Acquisition runs in a dedicated higher-priority task and writes complete samples
to a 512-entry RAM queue. The lower-priority output task batches up to eight
records per frame. At 100 Hz this queue represents about 5.12 seconds of
decoupling from a blocked transport. Queue overflow never overwrites an older
sample silently: sequence gaps and the cumulative lost-sample counter expose it.
decoupling from a blocked or disconnected transport. A failed write retains and
retries the same encoded packet rather than dequeuing more samples, so the queue
accumulates the outage backlog. After reconnection, the oldest retained data is
sent first. If the queue fills, acquisition drops new samples rather than
overwriting older ones; sequence gaps and the cumulative lost-sample counter
expose that permanent loss.
Receivers report bytes left in an incomplete trailing frame when capture ends.
Those bytes cannot pass CRC validation and are not silently admitted as samples.
@@ -11,6 +11,8 @@ data.
- Host compilation of the production C encoder with `-Wall -Wextra -Werror`
- Fragmented C-encoder-to-Python-parser contract test
- Deliberately corrupted CRC test with stream resynchronization
- Full eight-record, timestamp-saturation, invalid-size/count, and truncated-tail
encoder/parser contract cases
- ESP-IDF firmware build and flash on the assembled ESP32-C3 prototype
- Live USB capture followed by independent offline re-decoding
@@ -55,3 +57,26 @@ at priority 5 and receives samples through a 512-entry queue (about 5.12 seconds
at 100 Hz). The hardware capture's zero timing anomalies and zero loop overruns
confirm that packet encoding, CRC, float metadata, and USB output did not disturb
the acquisition cadence.
Output failure is transactional: firmware retains and retries the same encoded
packet with a scheduler delay instead of discarding it or dequeuing more samples.
The queue therefore accumulates a disconnected-transport backlog. If an outage
outlasts the queue, acquisition drops and counts new samples while preserving the
oldest queued data for ordered delivery after reconnection.
## Forced transport-outage validation
A temporary validation build made the packet writer report transport failure for
a fixed interval while acquisition continued normally. The failure injection was
removed before the production build.
With a three-second forced outage, all 2,144 observed samples arrived contiguously
from sequence 0 through 2,143. Packet gaps, sample gaps, drops, loop overruns, and
timestamp-saturation flags were all zero.
With a seven-second forced outage, the queue preserved samples 0 through 511 and
then dropped 138 new samples after reaching capacity. Delivery resumed at sample
650. The cumulative drop count and observed sequence gap both equaled 138. The
timestamp difference from sample 511 to 650 was exactly 1,390,000 us, matching
139 sample intervals, and no saturation flag was emitted. This verifies both the
oldest-data retention policy and the new exact timestamp re-anchor after overflow.