Symphony / Staging QA timed job

Nightly staging QA crawl

A report-only agent-browser route that crawls the staging product, validates the feature map, runs mutation probes, retests recent bugs, and creates focused follow-up tickets.

Cadence 02:00 Europe/LondonMode agent-browser reportScope staging tenant + feature mapOutput Linear + Slack + evidence

systemd timer->staging QA issue->agent-browser crawl->bugs + improvements

Operating model

timed job

The timer seeds a dated Linear issue with the qa:staging-regression label. Symphony routes it into a report-only policy, Codex uses agent-browser against the staging app, and the run completes only after report evidence, follow-up links, and Slack output are recorded.

60m

Crawl budget

default budget window

Code edits

inspect and report only

Mutation probes

contacts, companies, deals, tasks, assets, settings

Bug retest window

recent nightly QA findings

KEYTreat coverage drift as work

The crawl is not just a bug hunt. It compares live routes and controls against the app-owned feature map, then creates QA improvement tickets when the map or fixtures no longer describe the product.

System shape

click the nodes

The page follows the same timed issue pattern as the security agent. Click a node to trace how the timer turns into report evidence and improvement tickets.

Cadence design

compare modes

Each timed job has a small set of modes that decide whether the run is healthy, degraded, or ready to create follow-up work. Use the controls to compare the operating contract.

Preflight

Prove the crawl has a real target

The run checks agent-browser, auth, allowlisted domains, baseline data, and optional installed-app fixture seeding before it starts exploring.

agent-browser doctor and session check
dedicated QA account available
non-empty baseline tenant data
fixture seed path recorded

Feature coverage confidence82%

The crawl can close as a clean pass when evidence rows are complete.

Execution tracks

interactive

The tracks make each run inspectable. They separate inputs, outputs, and failure modes so reports do not collapse into vague status updates.

Mapped main pages

Feature coverage

Validate the app-owned feature map and record one action per row for every visible mapped feature.

Inputs

main-pages.json
live navigation
source routes

Outputs

coverage matrix
missing coverage tickets
skipped feature reasons

Coverage risk map

priority map

These are the surfaces the crawl should prioritize before spending remaining budget on deeper exploration.

Authentication and tenant datalogin state, roles, baseline records, cross-tenant guardrails

96

CRM mutation flowscreate, edit, reload, cleanup in core work areas

92

Deals and stage changespipeline state changes and persistence checks

84

Assets and uploads policyfile controls, links, previews, blocked upload actions

76

Settings save and revertaccount and org controls that must not leak or persist noise

70

Feature-map driftsource and live routes not represented by stable feature IDs

66

A clean visual crawl without mutation probes is not enough. When the run finishes early, unused budget should go into probes, recent-bug retests, and feature-map drift review.

Output rubric

ticket rules

The run should create follow-up work only when the output is concrete. Use the rubric to keep reports, bug tickets, and operational follow-ups separated.

Critical

Use for broken primary workflows, data loss, cross-tenant exposure, or a production-blocking regression visible on staging.

cannot sign in
cross-tenant data visible
record creation loses data

Run contract

evidence rows

The seeded issue body is the source of truth for run parameters and evidence. Completion is gated on workpad rows, not on a prose statement.

Issue seed

Timed issue

Title format is Nightly staging regression QA - YYYY-MM-DD with qa:staging-regression, staging, and frontend labels.

Allowed writes

Report-only route

No code edits, commits, pushes, PRs, or attempted fixes. Writes are report comments, Slack output, evidence attachments, and focused follow-up tickets.

Staging URL and allowed domains

Browser viewport and agent-browser session

Visible baseline data summary

Feature map path and validation result

Main-page feature matrix

Mutation probe summary with run tag

Recent bug retest summary

QA-system improvement review

Run-health retrospective

Slack summary and Linear report links

Local evidence cleanup after upload verification

Command plan

operator path

The implementation should start with the dry-run seeder and the agent-browser smoke checks before enabling the timer.

Dry-run issuesudo -u symphony /opt/symphony/elixir/scripts/create_staging_qa_issue.sh --dry-run --date 2026-05-21

Timer installsudo systemctl enable --now symphony-staging-qa.timer

List timersystemctl list-timers | grep symphony-staging-qa

Browser doctorAGENT_BROWSER_ARGS="${AGENT_BROWSER_ARGS:---no-sandbox}" agent-browser doctor --quick --json

!Keep generated media out of git

Screenshots, clips, traces, HARs, and dumps are temporary upload inputs. The durable evidence is the native Linear attachment URL after fetch verification.

Rollout path

ship safely

Dry-run the seeder no writes

Confirm title, labels, state, cycle, workpad rows, staging URL, allowed domains, budget, and viewport.

Seed account and fixtures auth first

Verify the dedicated QA account and baseline tenant. Optional installed-app fixture seeding is recorded when unavailable.

Enable nightly crawl report-only

Let the route create bug tickets and QA improvement tickets only when evidence is specific and fetchable.

Review drift weekly slow loop

Use repeated QA improvement tickets to tune fixtures, feature-map ownership, allowlists, and report quality.