Oncall agent
A docs-first design for an incident-to-fix oncall agent. Sentry detects the incident, Linear carries the complete context and workpad, Symphony dispatches Codex, and the fix ships through the existing PR and merge gates.
Operating model
tl;drThe oncall agent is not a separate production fixer. It is a precise intake and execution pattern: Sentry decides something needs attention, Linear makes the incident durable, and Symphony turns the ticket into a normal code-reviewable fix attempt. The design borrows Brex's strongest lesson: the useful unit is a structured incident report plus a continuously updated workpad, not a chat transcript.
Every useful fact lands in Linear before Symphony starts: Sentry links, impact, suspected owner, labels, runbook route, and a marked ## Codex Workpad. Codex can modify that workpad, open a PR, and drive the normal Symphony handoff. The ticket history becomes the learning loop.
Incident triggers
SentryV1 should start with Sentry issue alerts and leave metric or uptime triggers behind the same intake contract. The intake path can be a Sentry webhook receiver or a short poller. The important invariant is the same: normalize, redact, dedupe, then create one Linear ticket.
New or worsening error cluster
Best v1 trigger. Sentry issue alerts can fire on new issues, state changes, event volume, user impact, and project or environment filters.
- Sentry rule shape
- new issue, regression, events over threshold, users over threshold
- Linear output
- one Linear bug per normalized Sentry cluster
- Main risk
- duplicate tickets and noisy wrapper errors
System architecture
interactiveThe first implementation does not need to change Symphony's core scheduler. It adds an incident intake seam that creates rich Linear tickets, then lets the existing watched-state workflow do what it already does well.
Linear ticket contract
workpadThe seeder should create a ticket that is useful even before Codex starts. Each tab below is a required part of the incident issue description or metadata.
[Oncall] ac-python-api: POST /api/v1/crm/activities failing with ValidationError
Ready for Symphony
High when users are affected or events cross threshold
sentry, incident:oncall, repo:ac-python-api, sentry-id:API-123
The agent can update the marked workpad in the issue description. It should not rewrite the original Sentry facts section except to add links or corrections with evidence. This keeps the incident record reviewable.
Playbook stack
Brex patternBrex describes the hard part as turning oncall knowledge into maintainable context. For AgencyCore, that context should have three tiers so the prompt stays small but the agent can pull deeper material when needed.
Routing table
The smallest stable map from incident shape to context. It chooses repo labels, validation route, and first documents to read.
- Sentry project -> repo label
- top frame -> owner area
- endpoint -> domain prompt
- severity -> Linear priority
Sentry project to repo
ac-python-api maps to backend and API prompt context. ac-frontend maps to frontend validation and visual evidence rules.
Failure shape to docs
Auth, CRM, background jobs, and UI failures should each point at a short runbook plus the relevant repo-level instructions.
Ticket history to updates
Repeated root causes should update the routing table or runbook. Linear labels and Sentry IDs provide the retrieval key.
Fix-attempt lifecycle
interactiveThe agent is allowed to attempt a fix, but only inside the normal Symphony delivery path. Sentry is read-only until the PR is merged and merge-stage evidence exists.
Investigate
Symphony dispatches Codex. It reads the ticket, Sentry events, release context, repo code, tests, and runbooks before touching files.
root-cause hypothesis and evidence rows
bounded fix plan or explicit blocker
Guardrails and evals
trustThe eval target is not "did the model sound confident". It is whether the agent created a correct, bounded, reviewable incident ticket and then shipped a fix through the same controls as any other Symphony run.
Incident scope is one cluster
One ticket maps to one canonical Sentry cluster or one correlated metric incident. Families are cross-linked, not merged into a vague mega-ticket.
Facts and assumptions are separated
Sentry facts, repository evidence, and inferred hypotheses appear in distinct ticket sections so reviewers can challenge weak links.
Sentry stays read-only before merge
The agent can inspect Sentry during investigation. It resolves the issue only after merge verification in the existing merge-stage flow.
Linear is the durable memory
The marked workpad, report comments, PR links, and Sentry labels become the training set for future routing and runbook updates.
What v1 must prove
- Duplicate Sentry alerts do not create duplicate Linear tickets.
- The issue description contains enough sanitized context for a human to understand impact before Codex starts.
- Codex updates the marked workpad continuously and never loses the original incident evidence.
- Fix PRs run repo-routed validation and reach Human Review only with current evidence.
- Sentry resolution happens after merge verification, with the exact command recorded.