8d87f1c13d
Three audit items on the backup path. The script matched volume names by pattern and took the first hit, so a stale or restored volume could be backed up instead of the live one — and every verification step would then faithfully confirm the wrong database. It now asks the container what is mounted at /data and refuses ambiguity. Tested against a decoy volume that the old pattern would have matched first. check-images treated a photo as healthy if a file with the right name existed, so a truncated or partially restored file passed. It compares each file against the byte count its row records now; a one-byte stand-in for a 123KB photo is reported as WRONG SIZE and exits non-zero. The backup runs the same check against the stopped volume and exits 2 when the source was already damaged — still writing the archive, because a faithful copy of imperfect data is worth having, but saying so. The restart trap was installed after the app had already been stopped, so an interrupt in between could leave the service down with nothing to bring it back. The trap goes in first now, covers INT and TERM as well as EXIT, and records whether the container was running beforehand so a backup of an already-stopped app leaves it stopped. Verified on the live host: healthy source exits 0, damaged source exits 2 with the archive still written and verified, decoy volume correctly ignored, service answering immediately afterwards, and the test rows removed. Checks go from 290 to 294. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
273 lines
13 KiB
Markdown
273 lines
13 KiB
Markdown
# Parts Inventory
|
|
|
|
A catalog of what's actually on the shelf — components, filament, fasteners,
|
|
tooling — so "do I have one of those?" is a five-second search instead of an
|
|
hour of opening drawers.
|
|
|
|
Runs as a FastAPI + SQLite container on `mainserver` (`192.168.50.8`) behind
|
|
Caddy at **https://parts.tjm77.com**, alongside the other `~/srv` services.
|
|
|
|
## The data model
|
|
|
|
One `parts` table holds what every kind of stock has in common — name,
|
|
quantity, unit, location, cost, min-stock threshold. Everything type-specific
|
|
lives in `part_specs` as key/value rows, so a 0603 resistor, a spool of PLA and
|
|
a box of M3 screws share a table without a schema fork.
|
|
|
|
Categories and locations are both **nestable trees**. `Workshop / Bin A3` is a
|
|
real path, and filtering by `Workshop` returns everything in every bin beneath
|
|
it. Sidebar counts roll up the same way, so they always match what clicking the
|
|
filter actually returns.
|
|
|
|
Each category carries a **spec template** — the properties worth recording for
|
|
that kind of thing. Pick "Filament" on the add form and it pre-fills Material,
|
|
Colour, Diameter, Brand, Print Temp, and switches the unit to grams. That's what
|
|
keeps manual entry from being a blank page. Templates are suggestions only:
|
|
delete any row, add your own, ignore them entirely.
|
|
|
|
Every quantity change is written to `stock_log`, so a part's history answers
|
|
"where did those 40 headers go" instead of just showing a smaller number than
|
|
you remembered.
|
|
|
|
Parts carry **photos**. A bag falling apart after forty years still has the part
|
|
number printed on it, and the picture is worth more than any field you could
|
|
type it into. Add them from the part form — on a phone the picker opens the
|
|
camera directly — and they are downscaled in the browser before upload, which
|
|
keeps the server free of an image library. Files live beside the database in the
|
|
same volume; only metadata is in SQLite, because blobs there bloat the database
|
|
and complicate the backup. Uploads are sniffed by content rather than trusted by
|
|
their declared type, so nothing that claims to be a JPEG can come back out as
|
|
something a browser will execute; SVG is refused for the same reason.
|
|
|
|
Request size is capped in **two** places, because one is not enough. Starlette
|
|
parses and spools an entire multipart body before a route's dependencies run —
|
|
which means before the login check — so a route-level cap would only fire after
|
|
the bytes had already been written to disk by someone who doesn't know the
|
|
password. An ASGI middleware outside routing rejects an over-large body first,
|
|
and Caddy enforces the same ceiling at the edge.
|
|
|
|
Search is SQLite FTS5 over name, description, manufacturer, MPN, spec values,
|
|
tags, category and location, with prefix matching so results narrow as you type.
|
|
Typing `1.75mm`, `prusament`, `0603` or `Bin A3` all find the right things.
|
|
|
|
## Running it locally
|
|
|
|
```sh
|
|
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
|
|
PARTS_AUTH=off PARTS_DB=$PWD/data/parts.db .venv/bin/uvicorn app.main:app --port 8123
|
|
```
|
|
|
|
Then open http://127.0.0.1:8123.
|
|
|
|
## Tests
|
|
|
|
```sh
|
|
.venv/bin/python -m tests.test_api # 217 checks, in-process
|
|
.venv/bin/python -m tests.test_concurrency # 33 checks, against a real uvicorn
|
|
```
|
|
|
|
`test_api` exercises the API end to end against a throwaway database — the auth
|
|
gate and session-token signing, nested categories and locations, search across
|
|
every indexed field, filter rollups, stock adjustment and history, patch
|
|
semantics including explicit nulls, taxonomy cycle rejection, security headers
|
|
and asset versioning, login throttling on both the login and change-password
|
|
routes, and the full password-management flow including scrypt hashing, session
|
|
invalidation and the recovery CLI.
|
|
|
|
`tools/render_check.py` (43 checks) drives the real UI in a browser and
|
|
asserts what a person would see: that the JavaScript runs under the Content-Security-Policy,
|
|
that a wrong password says so, that picking "Filament" pre-fills its spec
|
|
template and switches the unit to grams, that the password section works, that a
|
|
photo uploads and comes back as a thumbnail you can open, and that nothing
|
|
overflows on a 390px phone.
|
|
|
|
```sh
|
|
.venv/bin/python -m playwright install chromium # once
|
|
.venv/bin/python -m tools.render_check --keep-shots /tmp/shots
|
|
```
|
|
|
|
It uses Playwright's own Chromium deliberately: headless Chrome driving the
|
|
installed browser returns nothing at all while a desktop Chrome is open, which
|
|
is silent enough to look like the app is broken.
|
|
|
|
`test_concurrency` needs a real server process, because a lost update only shows
|
|
up when two requests genuinely overlap inside SQLite. It fires overlapping
|
|
adjustments, patches, uploads and creates at one part and asserts the stock log always
|
|
sums to the stored quantity; races taxonomy renames against reads to check the
|
|
search index never describes a name the tree no longer has; and races session
|
|
revocations to check no epoch increment is lost.
|
|
|
|
## Configuration
|
|
|
|
The password is **managed from the app**, not from a config file. `PARTS_PASSWORD`
|
|
is only the bootstrap credential: it works until a password is set through the
|
|
UI, and is ignored from then on (otherwise "changing" the password would leave
|
|
the old one working). The stored password is a salted scrypt hash in the
|
|
`settings` table, which lives in the `/data` volume and survives rebuilds.
|
|
|
|
| Variable | Meaning |
|
|
|---|---|
|
|
| `PARTS_PASSWORD` | Bootstrap password, used only until one is set in the app. |
|
|
| `PARTS_SECRET` | Signing key for the session cookie. `openssl rand -hex 32`. |
|
|
| `PARTS_SESSION_DAYS` | Session lifetime, default 30. |
|
|
| `PARTS_AUTH` | `off` disables the login gate (LAN-only use). |
|
|
| `PARTS_DB` | SQLite path. `/data/parts.db` in the container. |
|
|
| `PARTS_IMAGE_DIR` | Where photos are written. Defaults to `images/` beside the database. |
|
|
| `PARTS_LOGIN_MAX_FAILURES` | Failed logins allowed per window, default 10. |
|
|
| `PARTS_LOGIN_WINDOW` | Throttle window in seconds, default 300. |
|
|
| `PARTS_SECURE_COOKIE` | `auto` (default) trusts `X-Forwarded-Proto`; `on`/`off` force it. |
|
|
|
|
Every token carries a session epoch. Changing the password bumps it, which
|
|
invalidates every outstanding cookie at once while re-issuing one for the
|
|
browser that made the change — so a password change really does sign out other
|
|
devices, with no need to touch `PARTS_SECRET` on the host. **Sign out other
|
|
devices** in Settings bumps the epoch on its own.
|
|
|
|
Failed logins are throttled on a **global** window rather than per source
|
|
address. Caddy appends to `X-Forwarded-For` instead of replacing it, so the
|
|
client-supplied end of that header is forgeable and a per-IP bucket would be
|
|
trivially evaded. There is one legitimate user, so a global cap costs nothing
|
|
real; the trade-off is that a guessing spray can block the login form for the
|
|
window. Existing sessions keep working throughout.
|
|
|
|
## Deploying
|
|
|
|
Lives at `/home/jay/srv/parts/` on `.8` and follows the same conventions as the
|
|
other services there — `build: .`, external `caddy_web` network, named volume
|
|
for `/data`.
|
|
|
|
`~/srv/parts` is a checkout of this repository — `git log` there tells you
|
|
exactly what is deployed — but **deploys go over rsync, not `git pull`**:
|
|
|
|
```sh
|
|
rsync -az --delete \
|
|
--exclude '.venv' --exclude 'data' --exclude '__pycache__' --exclude '*.pyc' \
|
|
--exclude '.env' --exclude '.env.bak-*' \
|
|
./ jay@192.168.50.8:/home/jay/srv/parts/
|
|
ssh jay@192.168.50.8 'cd ~/srv/parts && docker compose up -d --build'
|
|
```
|
|
|
|
`.env` is excluded from the transfer, and rsync's `--delete` leaves excluded
|
|
files alone, so the deployed credentials survive.
|
|
|
|
The reason it isn't `git pull`: **gitea on `.8` cannot serve a clone or fetch to
|
|
`.8` itself.** Auth succeeds and gitea logs `git-upload-pack ... 200 OK`, then
|
|
the transfer dies with `fetch-pack: unexpected disconnect while reading sideband
|
|
packet`. Reproduced against both `git.tjm77.com:2222` and `127.0.0.1:2222`, with
|
|
`--depth 1`, with protocol v0, and with fsck disabled; `git ls-remote` succeeds
|
|
every time, and cloning the same repo from a laptop works. Undiagnosed. Until
|
|
it's fixed, the checkout in `~/srv/parts` is placed there by rsyncing the
|
|
working tree *including* `.git`, which is why `git status` there is clean.
|
|
|
|
`~/srv` (the infra repo) ignores `parts/`, since this directory is its own
|
|
repository.
|
|
|
|
The database is in the `parts_parts_data` docker volume, which survives
|
|
rebuilds.
|
|
|
|
```sh
|
|
./tools/backup.sh /path/to/backups
|
|
```
|
|
|
|
A backup is **two** resources that reference each other — the database and the
|
|
photo files — so capturing them at different moments is not a backup. A photo
|
|
deleted between the two steps leaves the saved database pointing at a file the
|
|
archive doesn't contain; one added leaves the reverse. Neither is repairable
|
|
afterwards, and no amount of pruning fixes the direction that lost data.
|
|
|
|
So the script stops the app for the few seconds the copy takes. With no process
|
|
attached, `parts.db` and its `-wal`/`-shm` sidecars are a consistent set (which
|
|
is also why `cp parts.db` alone is wrong on a running database — recent commits
|
|
may still be sitting in the WAL) and the images directory cannot move
|
|
underneath.
|
|
|
|
It resolves the data directory by asking the container what is mounted at
|
|
`/data`, rather than matching volume names by pattern — a stale or restored
|
|
volume with a similar name would otherwise be backed up instead, and every
|
|
check would then faithfully verify the wrong database. Ambiguity is refused
|
|
rather than guessed at.
|
|
|
|
Then it verifies three things: that the source data is internally consistent,
|
|
that every photo the database references is present in the archive, and that
|
|
the archived database opens and passes SQLite's `integrity_check`. Exit status
|
|
is `0` when all of that holds, `1` if the archive is incomplete or unusable,
|
|
and `2` if the archive is fine but the *source* was already damaged — you still
|
|
get the backup in that case, because a faithful copy of imperfect data is worth
|
|
having; you just get told.
|
|
|
|
Restore by stopping the container and unpacking the archive into the volume.
|
|
|
|
Two integrity commands, neither of which is a routine step:
|
|
|
|
```sh
|
|
docker exec parts python -m app.admin check-images # drift, in both directions
|
|
docker exec parts python -m app.admin prune-images # delete files nothing references
|
|
```
|
|
|
|
`check-images` compares each file against the byte count its row records, so a
|
|
truncated or partially restored photo is caught rather than waved through on
|
|
the strength of its filename. It exits non-zero for a referenced photo that is
|
|
missing or the wrong size — that is data loss, where a stray file is just
|
|
clutter. `prune-images` ignores
|
|
anything less than an hour old, because an upload writes its file before
|
|
inserting its row and a young orphan is indistinguishable from an upload still
|
|
in flight; `--all` overrides that and is only safe with the app stopped.
|
|
|
|
## API
|
|
|
|
Everything under `/api` is JSON and cookie-authenticated. `GET /healthz` is open.
|
|
|
|
```
|
|
GET /api/parts?q=&category_id=&location_id=&tag=&low_stock=&sort=&limit=&offset=
|
|
POST /api/parts
|
|
GET /api/parts/{id}
|
|
PATCH /api/parts/{id}
|
|
DELETE /api/parts/{id}
|
|
POST /api/parts/{id}/adjust {delta, reason}
|
|
GET /api/parts/{id}/history
|
|
GET /api/parts/{id}/images
|
|
POST /api/parts/{id}/images multipart: file, caption
|
|
GET /api/parts/{id}/images/{token}
|
|
PATCH /api/parts/{id}/images/{token} {caption, position}
|
|
DELETE /api/parts/{id}/images/{token}
|
|
GET /api/categories POST /api/categories PATCH|DELETE /api/categories/{id}
|
|
GET /api/locations POST /api/locations PATCH|DELETE /api/locations/{id}
|
|
GET /api/tags
|
|
GET /api/stats
|
|
```
|
|
|
|
The "what could I build with what I'm holding?" idea is meant to arrive as
|
|
another consumer of these endpoints — the arbiter on `.8` already has the lane
|
|
routing and typed-action machinery for it — rather than as a fork of them.
|
|
|
|
## Passwords
|
|
|
|
Open **Settings** (the gear in the header) and use the Password section. It asks
|
|
for the current password, takes a new one twice, and signs out every other
|
|
device. Until a password has been set in the app, a banner says so.
|
|
|
|
The only reason to touch a terminal is a forgotten password:
|
|
|
|
```sh
|
|
docker exec -it parts python -m app.admin set-password # prompts, twice
|
|
docker exec parts python -m app.admin show-status # where the password comes from
|
|
docker exec parts python -m app.admin clear-password # fall back to PARTS_PASSWORD
|
|
```
|
|
|
|
Each of those signs out every session too.
|
|
|
|
## Hardening notes
|
|
|
|
Responses carry `X-Content-Type-Options`, `X-Frame-Options`, `Referrer-Policy`
|
|
and a `Content-Security-Policy` that keeps the page to same-origin scripts and
|
|
no framing; HSTS is added when `X-Forwarded-Proto` says the original request was
|
|
HTTPS. The session cookie is HttpOnly, SameSite=Lax and Secure, and its
|
|
signature is a fixed-width HMAC appended to the payload rather than delimited —
|
|
a delimiter byte can occur inside a raw digest, which previously invalidated
|
|
about 12% of issued tokens.
|
|
|
|
Static assets are served under content-hashed URLs (`app.js?v=<sha>`), so
|
|
Cloudflare caching them for hours is harmless: a deploy changes the URL. The
|
|
bare, unhashed paths are served `no-cache` so nothing can pin stale frontend
|
|
code against a newer API.
|