QA architecture

Reference for how supportsn/sn-automation-testing works under the hood. The Automation testing SOP is the runbook; this page is the map. Everything runs on free tiers: GitHub Actions runner + Playwright, Cloudflare Pages / Workers / KV / R2 free, and the Claude subscription token for the agentic engine (no API billing).

System at a glance

GitHub Actions (free runner)                Cloudflare (free tier)
┌─────────────────────────────┐             ┌──────────────────────────────┐
│ qa-portfolio  (SEO, daily)  │──verdicts──▶│ git → Pages rebuild          │
│ qa-uptime     (TLS/DNS, 6h) │──verdicts──▶│   qa-dashboard (static)      │
│ qa-design     (manual)      │──images───▶ │ R2  qa-visual (live, no      │
│   design verdict → git      │             │     rebuild)                 │
│ qa-agentic    (manual)      │──verdicts──▶│ KV  UPTIME_KV                │
└─────────────────────────────┘             │ Worker (cron 15min + API)    │
                                            └──────────────────────────────┘

Five engines emit one shared verdict schema, one file per source: <siteId>.<source>.json where source ∈ tier1 (SEO) / design / visual / uptime / agentic (+ content from project repos). Each verdict carries generatedAt and runUrl (the GitHub Actions run that produced it).

Hosts

Host	Runs	Why there
GitHub Actions (free runner)	SEO sweep, design+visual capture, GHA uptime, agentic	full Node + Playwright Chromium, no render rate limits, 2 000 free min/month
Cloudflare Worker (`uptime-worker/`, deployed via Workers Builds from the repo)	15-min uptime cron, daily-summary cron, live APIs, baseline promotion	edge cron can run every 15 min at zero GHA cost; no auto-billing on free
Cloudflare Pages	the dashboard (Astro static) + Pages Functions (`trigger`, `settings`, `history`, `active-runs`)	free static hosting; rebuilds on push (git-as-data)

Data stores — the hybrid model

Store	Holds	Why
git (`qa-dashboard/src/data/`)	SEO / design / uptime / agentic verdicts (small JSON)	git-as-data: commit → Pages rebuild → auditable history for free
R2 (`qa-visual` bucket)	visual screenshots (`visual/<site>/<stem>--{baseline,current,diff}`), verdict (`visual/<site>/latest.json`), canonical baselines (`baselines/<site>/<stem>.jpg`), provenance (`meta.json`, `drift.json`)	images are heavy and change every run — keeping them in git bloated the repo and burned Pages builds; R2 has free egress and the dashboard reads it live
KV (`UPTIME_KV`)	`latest` uptime blob, `days` (30-day per-site aggregates), `incidents` (transition log), `rollup` (daily-summary accumulator)	one combined blob per write keeps well under the 1 000 writes/day free cap

Visual data flow (the R2 pipeline)

qa-design.yml pulls baselines from R2 (canonical; git copies only seed sites R2 doesn’t have yet).
The runner captures full-page JPEGs, diffs against the baselines, writes provenance (meta.json = when/which run captured each baseline; drift.json = when drift was first seen — kept until stable or promoted).
Everything visual uploads to R2; only the small design verdict is committed (no rebuild for visual, no image bloat).
The dashboard’s site page fetches WORKER/api/visual/<site> and streams images from WORKER/visual/<site>/<file> — results are live seconds after the run.
Promote baseline = POST WORKER/api/promote-baseline — the worker copies the captured “New” over the baseline in R2, stamps provenance (promoted), closes the drift window, and flips the verdict frames to stable. No re-run, no commit.

If R2 credentials are missing, the workflow falls back to committing visual to git (the old behaviour), so nothing breaks.

The edge worker

uptime-worker/ — name sn-automation-testing, deployed automatically by Workers Builds on push (root directory uptime-worker, build watch paths uptime-worker/* + qa/sites.json so routine data commits don’t trigger builds).

Cron */15 — fetch every site once (pool of 6 — the free tier’s concurrent connection limit, so queue wait isn’t counted as latency; real browser UA because Duda throttles bot agents; slow threshold 8 s). Writes one combined KV blob, updates the 30-day aggregates, logs incidents on transitions, Slack-alerts on down/recovered only (no spam).
Cron 55 16 * * * (23:55 ICT) — end-of-day Slack summary (checks run, outages, slow sites) from the daily rollup, then resets it.
HTTP API (CORS-open, so the dashboard reads it live): /uptime.json, /api/history, /api/incidents, /api/visual/:site, /visual/:site/:file, POST /api/promote-baseline, plus manual /run, /daily-report, and / (its own live status page).

Worker checks are HTTP-liveness only — a Worker has no raw TLS socket or DNS resolver, so TLS-expiry + DNS stay on the GHA 6-hour job. The two are complementary by design.

Scheduling — the catch-up gate

qa/scripts/due-sites.mjs (dependency-free) reads the dashboard schedule (settings.json: off / hourly / daily (02:00 UTC) / weekly (Mon 02:00), global + per-site) and each site’s last committed generatedAt. A site is due when it hasn’t been checked since the current cadence slot opened — so when GitHub delays or drops the 02:00 cron (it often does), the next hourly run catches up instead of silently skipping the day. Manual workflow_dispatch always bypasses the gate.

Cost budgets (free-tier ceilings to respect)

Budget	Ceiling	Current use / rule of thumb
GitHub Actions (private repo)	2 000 min/mo	uptime ~120 · SEO daily fleet ~600 · design/visual fleet run ≈ 130 min per full-fleet pass → keep visual manual/weekly or batch it
Cloudflare Pages builds	500/mo	only verdict commits rebuild (~1–2/day); visual doesn’t rebuild at all
Workers free	100 k req/day, 1 000 KV writes/day, 50 subrequests/invocation	cron+dashboard ≈ a few hundred req/day; 1 fetch/site (31 < 50); ~300 KV writes/day
R2 free	10 GB, free egress	latest+baseline only ≈ 1–2 GB, overwritten in place
Claude quota (agentic)	subscription 5-h window	~$0.7–1.2 equiv/site, ~5.5 min; 5 parallel verified; fleet pass ≈ $30 equiv — run selectively

Secrets

Secret	Where	Used for
`GITHUB_TOKEN` (built-in)	workflows / Pages Functions (`GH_REPO` too)	committing verdicts, workflow dispatch from the dashboard
`R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY`	repo secrets	`qa-design.yml` upload/sync with the `qa-visual` bucket
`SLACK_WEBHOOK_URL`	worker secret and repo secret (same webhook)	down/recovery alerts + daily summary (worker); TLS/DNS alerts (GHA)
`CLAUDE_CODE_OAUTH_TOKEN`	repo secret	agentic runs on the Claude subscription (no API billing)
`SEO_NAV_MCP_TOKEN`	repo secret	the agentic content oracle (SEO Navigator MCP)

Never commit secret values; local scratch files (.dev.vars, tmp.txt) are gitignored.

Repo map

qa/                     the platform-agnostic kit
  src/core/             seo-auditor · design-auditor · visual-diff · sitemap · report
  src/run-*.ts          portfolio (SEO) · design (+visual) · uptime · content-fidelity
  standards/            design-checklist.json — the machine-readable standard
  scripts/due-sites.mjs the catch-up schedule gate
  sites.json            the fleet (31 sites)
  baselines/<site>/     git seed copies (R2 is canonical)
.github/workflows/      qa-portfolio · qa-design · qa-uptime · qa-agentic · qa-reusable
qa-dashboard/           Astro 5 + Tailwind 4 static dashboard + Pages Functions
uptime-worker/          the edge worker (cron + APIs + R2/KV bindings)
.claude/skills/qa-verify/  the agentic QA skill
docs/ARCHITECTURE.md    in-repo deep dive