thejayman77
a56cea6e2a
Four audit findings on the photo feature. The upload route was async, so its blocking SQLite work ran on the event loop: under contention it stalled every other request for SQLite's busy timeout, not just its own. It also read the photo count without the write lock, so overlapping uploads all observed the same total and stored past the ceiling together. It is a synchronous endpoint now, running in the threadpool, taking BEGIN IMMEDIATE before re-checking the part, the ceiling and the position, and committing before it returns. Reverting either half makes the new test die with the same TimeoutError the audit reported. The 8MB cap protected nothing: Starlette parses and spools an entire multipart body before a route's dependencies run — before the login check — so the bytes were already on disk by the time anything rejected them, and an anonymous caller could make us write them. A plain ASGI middleware outside routing now refuses an over-large body first, and Caddy enforces the same ceiling at the edge. The documented backup captured the database and the photos at two different moments while the app stayed writable, so a photo deleted in between left the saved database pointing at a file the archive did not contain. tools/backup.sh stops the app for the few seconds the copy takes and verifies afterwards that every referenced photo is in the archive. Cleanup could destroy data rather than merely litter: prune-images could delete a file between an upload writing it and inserting its row, and deletions unlinked before their transaction committed. Pruning now ignores anything under an hour old unless forced, and deletes commit before unlinking — an orphaned file is recoverable, a row without its photo is not. check-images reports drift in both directions and fails only on the direction that loses data. Checks go from 271 to 291. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
49 lines
2.1 KiB
Bash
Executable File
49 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Consistent backup of the parts inventory.
|
|
#
|
|
# The database and the photos are two resources that reference each other, so
|
|
# capturing them at different moments is not a backup: a photo deleted between
|
|
# the two steps leaves the saved database pointing at a file that isn't in the
|
|
# archive, and one added leaves the reverse. Neither is repairable afterwards.
|
|
#
|
|
# So the app is stopped for the few seconds it takes to copy both. With no
|
|
# process attached, parts.db and its -wal/-shm sidecars are a consistent set and
|
|
# the images directory cannot move underneath us.
|
|
#
|
|
# ./tools/backup.sh [output-directory]
|
|
set -euo pipefail
|
|
|
|
OUT_DIR="$(cd "${1:-$PWD}" && pwd)"
|
|
cd "$(dirname "$0")/.."
|
|
STAMP="$(date +%F-%H%M%S)"
|
|
NAME="parts-backup-$STAMP.tar.gz"
|
|
VOLUME="$(docker compose config --format json 2>/dev/null \
|
|
| python3 -c 'import json,sys; print(list(json.load(sys.stdin)["volumes"])[0])' 2>/dev/null || echo parts_data)"
|
|
VOLUME="$(docker volume ls --format '{{.Name}}' | grep -E "parts.*data" | head -1)"
|
|
|
|
echo "Stopping the app so both halves are captured at one moment..."
|
|
docker compose stop parts >/dev/null
|
|
|
|
restart() { docker compose start parts >/dev/null && echo "App restarted."; }
|
|
trap restart EXIT
|
|
|
|
# What the database expects to exist, read while nothing can be writing.
|
|
EXPECTED="$(docker run --rm -v "$VOLUME":/data -e PARTS_DB=/data/parts.db \
|
|
--entrypoint python parts -m app.admin list-image-files | sort)"
|
|
|
|
docker run --rm -v "$VOLUME":/data -v "$OUT_DIR":/out alpine \
|
|
tar -czf "/out/$NAME" -C /data .
|
|
|
|
# Verify every referenced photo actually made it into the archive.
|
|
IN_TAR="$(tar -tzf "$OUT_DIR/$NAME" | sed -n 's#^\./images/##p' | sort)"
|
|
MISSING="$(comm -23 <(printf '%s\n' "$EXPECTED" | grep -v '^$' || true) \
|
|
<(printf '%s\n' "$IN_TAR" | grep -v '^$' || true) || true)"
|
|
if [ -n "$MISSING" ]; then
|
|
echo "BACKUP INCOMPLETE — referenced photos missing from the archive:" >&2
|
|
printf ' %s\n' $MISSING >&2
|
|
exit 1
|
|
fi
|
|
|
|
COUNT="$(printf '%s\n' "$EXPECTED" | grep -c . || true)"
|
|
echo "Wrote $OUT_DIR/$NAME ($(du -h "$OUT_DIR/$NAME" | cut -f1)), $COUNT photo(s) verified present."
|