Files
on_the_level/AUDIO_DESIGN_LOG.md
T
thejayman77 36c0cfa5ab Audio: Surface proximity ticking + Voice settled-change; stage Edge staircase
Replaces the hands-free audio assist after extensive on-device iteration
(full journey + rejected approaches in AUDIO_DESIGN_LOG.md; current design
in SOUND_DESIGN_HANDOFF.md, from Jay's sound-design sessions).

Surface (TONES) = continuous proximity ticking: tick.wav repeats faster as
you near level (axis-agnostic warmer/colder, the rate is the message).
- Exponential rate in stable tilt error, clamped 1.5/s (>=~5 deg) to 8/s
  (<=~0.4 deg), hard-capped; constant loudness (acceleration is the signal).
- Deadline scheduler off a monotonic clock (SystemClock.elapsedRealtime):
  advances a next-tick deadline, so no drift and no catch-up bursts; <=1 tick
  per sensor update; immediate tick on re-entry (enable/return FACE_UP/unlock).
- Lock reuses LockDetector.isLocked (dwell + hysteresis, no second audio
  threshold): stop ticking, play level.wav once, silence while held. The
  1.39s level stream is retained and stopped on unlock/disable so a fresh
  tick can't overlap its tail.
- Pure SurfaceTickPolicy (unit-tested: anchors/clamps, monotonic rates,
  gating, immediate re-entry, no catch-up burst, level-once, resume on unlock).

Voice (VOICE) = settled-change announcer: one correction on the dominant
axis, silent while moving, speaks again on settle only if direction changed,
crossed coarse->fine, or reached lock. No periodic repeat. (VoiceGuidancePolicy,
unit-tested.)

Assets: tick.wav + level.wav downsampled to mono 44.1kHz in res/raw (from
Jay's SoundQ-derived 96k masters). AudioAssistMode adds OFF/TONES/VOICE.

Retired the settle-gated two-ding packet scheduler (AudioAssistStateMachine +
its ding assets) - superseded by ticking for Surface; logged as tested/rejected.

Edge staircase (1-D, future): 5 pitch-contour masters staged in
sonar-staircase-v3/ for when Edge mode is built.

76 tests passing; assembleDebug clean; blessed on-device.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 09:43:18 -04:00

4.9 KiB

Audio Assist — design log

Hands-free audio to level something without watching the screen. Setting cycles OFF / TONES / VOICE from the Level header. Plays on USAGE_MEDIA (so the volume slider controls it) and never takes audio focus.

Source of truth for the current design is [SOUND_DESIGN_HANDOFF.md], from Jay's dedicated sound-design sessions. This log records the journey — especially rejected approaches — so they don't get rediscovered. Where the two ever disagree, the handoff wins.

Chosen direction (current)

Two modes / two mechanisms, one sound identity (all derived from a 500 Hz blip):

SURFACE (2-D) — TONES = continuous proximity ticking. A tick (tick.wav, 60 ms) repeats faster as you near level — axis-agnostic "warmer/colder", the RATE is the whole message (a bubble level asks nothing of you). No settle gate — it ticks at every point in the adjustment. Rate is exponential in the stable tilt error, clamped 1.5/s (≥~5°) → 8/s (≤~0.4°), hard-capped at 8/s. Constant loudness (acceleration is the signal, not volume). On lock (reusing LockDetector.isLocked — dwell + hysteresis, no second audio threshold): ticking stops, level.wav plays once, silence while it holds; unlock resumes ticking. Scheduling is deadline-based off a monotonic clock — advances a next-tick deadline (no drift, no catch-up bursts, ≤1 tick per sensor update); re-entry (enable TONES / return FACE_UP / unlock) ticks immediately. level.wav's 1.39 s stream is retained and stopped on unlock/disable so a fresh tick can't overlap its tail.

EDGE (1-D, future) — the "sonar staircase." Direction + magnitude as a 4-blip pitch contour (rising = raise it, falling = lower it, steepness = how far; level sounds flat), one report per move→settle. Five pre-rendered assets staged in sonar-staircase-v3/; wired when Edge mode is built.

VOICE — deliberately separate, a SETTLED-CHANGE announcer (not a timer). Speaks one correction, then stays silent while moving; when it settles again it speaks only if the dominant direction changed, it crossed coarse→fine, or it reached lock ("that's level"). A settle still needing the same coarse nudge gets silence. No periodic repeat, no bed under it.

Implementation: pure SurfaceTickPolicy (monotonic deadline scheduler, exponential rate, unit-tested) + VoiceGuidancePolicy; the SoundPool layer stays dumb (play a tick / play & stop level). Sounds are Jay's SoundQ-derived blips (tick.wav, level.wav) downsampled to mono 44.1 kHz in res/raw.

Rejected approaches (and why)

  1. Two fixed proximity bands + lock ping (original). Worked but coarse; the near/close jump was audible ("a lower ding when close").
  2. Continuous cadence — beep interval shrinks smoothly toward center (parking-sensor). → "gets a bit much." Monotonous over a long adjustment.
  3. Rising-pitch discrete beeps. Better, but still same-note-ish and busy.
  4. Continuous glide tone — one warm tone, pitch low→treble by closeness, resolving into the aura. → "the constant sound works but it's too much; not clicking." A continuous tone gives the ear no silence to rest on, so it fatigues; it also bypassed the tested scheduler.
  5. Radar doublet on a per-band cadence — two sonar pings ~110 ms apart, repeating on a 1.3/0.9/0.6 s cadence while off-level. The literal two-ping "pew-pew" read as UI/radar beeps, not sonar, and the continued cadence was still a clock. Kept the bands/pitch, dropped the cadence in favor of one packet per settle.
  6. Continuous forcefield aura bed (looping CC0 ambience while locked). Seamless on a PC media player but clicked at the loop point through Android SoundPool. Pulled; a gapless bed needs AudioTrack/ExoPlayer.
  7. Settle-gated two-ding "status packet" (banded pitch + gap, one packet per move→settle, fuller arrival ding at lock). Built, unit-tested, on-device tested. Coherent, but on the bench the settle-then-report model felt sparse/laggy for the fiddly 2-D surface adjustment — you want live "warmer/colder" while you're actually moving it, not a report after you stop. Superseded by continuous ticking for Surface. (The settle-then-report shape lives on for VOICE and for future EDGE, where a discrete per-settle report fits the 1-D task.)

Guiding principles (learned)

  • Leave air between cues; silence is restful. (600 ms near cadence is responsive without being alarm territory; ~180 ms was alarm territory.)
  • Pitch carries distance more reliably than volume (phone volume + room noise make loudness a poor information channel).
  • Keep the SoundPool layer dumb; put decisions in tested pure logic.
  • The centered sound needs source loudness/spectrum presence, not just a bigger volume number.
  • No reverb/comb tails on phone speakers — they smear and lengthen fatigue.
  • Reuse the real lock state for the centered handoff; never invent a second audio threshold.