The deploy pipeline runs from the working tree, so a wave of shipped features
had never been committed. This snapshots git to what's actually running.
SEO impression recovery (live + verified):
- Duplicate /a/{id} now 301-redirect to their canonical twin instead of 404
(a hard 404 silently dropped already-indexed URLs and tanked impressions).
- Dedup representative selection reworked: accepted/serveable -> established
rep (URL stability) -> quality score, so an accepted page never retires to a
rejected rep and an indexed canonical doesn't churn when a newer twin arrives.
- HEAD /a/{id} returns the same status as GET (api_route GET+HEAD) instead of
falling through to the static mount and 404ing.
- `dedup --force-recluster`: cycle-locked, model-free re-cluster to re-apply the
policy to the existing corpus (shared cycle_lock context manager).
- CLI honors GOODNEWS_DB for its default --db (was silently ignored).
Publishing Desk (admin tool to post highlights to X via Web Intents):
- publishing.py queue/rank/handle-resolution; admin UI; full searchable emoji
picker (bundled data, no CDN) for the blurb editor.
Play games + site:
- Bloom (word-wheel), Memory Match, daily ritual set, Zen Den (dev-gated).
- English-only language gate; source prospecting; paywall + dedup hardening.
Tests: full suite green (349). Ignores tightened (node_modules, data/*.db).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two things found while chasing the recurring ~15min slowness:
- dedup.py: cluster_duplicates re-ran an O(n²) cosine pass over ALL ~3.7k
articles and rewrote duplicate_of for every one of them EVERY cycle — even
when nothing new arrived (embedded=0) — ~53s CPU + a large WAL commit that
starved live API reads (/api/brief 2-7s). Now skip the re-cluster entirely
when nothing new was embedded (clusters can't have changed). Verified: cycle
drops from ~53s to ~1s and /api/brief stays at 20ms through a cycle, vs 2-7s
before. (A real new article still triggers a full re-cluster.)
- games.py _build_grid: word placement took the first random valid spot, so
words rarely crossed. Now gather valid placements and PREFER ones that cross
an already-placed word (shared matching letter), falling back to any valid
spot — so the grid interlocks like a real word search. Every word still
placed (tests green). NOTE: changes today's grid layouts, so an in-progress
word search resets once.
Also added a systemd drop-in (Nice=19/CPUWeight=20/IOWeight=10/ionice-idle) to
deprioritize the batch cycle — minor, the dedup skip is the real fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- LocalModelClient.embed() calls the OpenAI-compatible /embeddings endpoint
(local nomic model); base_url shared with chat, model via GOODNEWS_EMBED_MODEL.
- New article_embeddings table and articles.duplicate_of column (+ migration).
- dedup module: embeds missing articles, clusters near-identical stories within
a date window by cosine similarity (pure-stdlib, vectors normalised once), and
marks all but the highest-ranked member of each cluster as a duplicate.
- 'dedup' CLI command; cycle now runs poll -> classify -> dedup -> brief.
- Feed and brief queries hide duplicates, so a story carried by multiple
outlets shows once.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thejayman77