thejayman77 ee43bb0df6 analytics: filter known-bot User-Agents at /api/events (honest visitor counts) ...

Many modern crawlers (AI scrapers, headless Chrome, link-preview fetchers) run JS and
fire the visit/summary_viewed beacon, inflating "visitors" even though there's no
human discovery channel. Apply queries.is_bot_ua() at /api/events — the same filter
the load-error beacon uses — so honest bot UAs (GPTBot, AhrefsBot, headless Chrome,
python/curl, …) are dropped before recording. Response is identical so a bot can't
detect it. Counts read lower but truer going forward (past rows unchanged). Won't catch
UA-spoofing bots; that needs a heavier heuristic. Tests: bot UAs dropped, real browser
counted; existing event tests send a real UA (default client UA contains "python").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-30 11:19:51 -04:00

39 KiB

800x394px

Raw History