The Agent Execution Stack
How AgencyCore runs AI agents: durable Inngest workflows composing a webhook-driven agent runtime over a pluggable execution backend. The system at the level of architecture, lifecycle, and the core components.
At a glance
tl;drTwo subsystems hold this up. inngest_functions is the durable composition layer: event-triggered step functions that orchestrate AI work. agent_runtime is the atom underneath: a provider-neutral library that runs exactly one agent. The composition is durable; the backend is pluggable.
Agent Runtime knows how to run one agent and nothing about workflows. Everything above is composition: Agent Runner makes a run durable, Components batch runs into a capability, Workflows sequence capabilities into a product. And because Agent Runtime is an abstraction, not Anthropic-specific, Claude Managed Agents is just the current implementation behind the interface.
System design — the four tiers
interactiveRead top to bottom: from what a client calls down to where the work happens. Each tier hides the one below it. Click a box to trace what it connects to. The dashed line is the swap point — below it, the backend is replaceable.
The Core Service layer is itself a stack. Agent Runtime runs one agent; Agent Runner makes it durable; Components batch it; Workflows sequence it. Each layer adds exactly one thing.
Agent Runtime is an interface, not a vendor. Managed Agents satisfies the provider contract today; a background-worker backend that satisfies the same contract drops in with nothing above it changing.
Request lifecycle
a Sonar runOne company-discovery run as a sequence across the four tiers, time flowing down. Solid arrows are calls; dashed arrows are async events and the SSE stream the client waits on. It plays on a loop — click any step to pin it.
Webhook-driven execution
signature The mechanic the whole stack rests on: a multi-minute agent run holds no worker. The runner parks on a durable wait; the backend finishes whenever it finishes and a signed webhook wakes the exact parked step by run_id.
poll_status on every wake, so the run still converges if a webhook never arrives. Agent Runner — the durable bridge
core component Agent Runtime is a plain async library: open_run() returns once the session is open, but the answer arrives minutes later over a webhook. Agent Runner converts that into a durable, replay-safe Inngest step graph that retries deterministically and never double-bills. It exposes three entry points.
| Function | Shape | Used by | Failure policy |
|---|---|---|---|
run_agent_output | 1 required planner | Workflows | permanent → NonRetriableError aborts workflow |
run_agent_batch | N subagents, parallel | Components | permanent isolated per-branch; siblings continue |
agent_run | standalone queued run | Runs API | on_failure closes run after retries |
Components run on run_agent_batch: one deterministic prefetch, then a failure-isolated batch of subagents that fans back into one aggregated result. One branch below fails permanently — watch it isolate while its siblings still land.
Agent run state machine
interactive Every agent_runs row walks this graph; all terminal writes are status-gated, so a webhook and a reaper racing to close the same run resolve to one outcome. Click a state to trace its transitions.
Error taxonomy
interactiveThe runner translates the runtime’s neutral provider errors into Inngest retry decisions. The same error is handled differently on the single path versus inside a parallel batch. Pick a class.
A malformed request or unsupported operation will never succeed on retry. On the single path it raises NonRetriableError, which aborts the workflow immediately rather than burning the retry budget.
Configuration & the dispatch gate
interactiveINNGEST_WORKFLOW_DISPATCH_SLUGS is the rollout lever: per app, it routes a workflow to the new Inngest engine or the legacy Celery path. Both produce the same workflow_runs row and the same SSE frames — only the engine differs. Toggle to compare.
When the workflow’s slug is enabled for this environment, dispatch sends an Inngest event and the durable function drives the run.
Operations
run it Three things must be wired for the stack to run end-to-end, plus one debugging path for the most common failure — a run stuck at running.
Register the Anthropic webhook
Point the Console session webhook at /api/v1/agent-runtime/anthropic/webhook and set ANTHROPIC_WEBHOOK_SIGNING_KEY. Boot fails in production without it — the receiver cannot verify signatures, so no run ever finalizes.
Wire the Inngest app
Set INNGEST_EVENT_KEY + INNGEST_SIGNING_KEY (required in prod) and register the app so Inngest Cloud can reach /api/inngest. INNGEST_WORKFLOW_DISPATCH_SLUGS gates which workflows take the new path.
Watch the crons
reap-stale-agent-runs (every 15m) expires runs past deadline; aggregate-daily-ai-usage (00:05 UTC) rolls up billing. Both are Inngest TriggerCron functions, status-gated and idempotent.
Debug a stuck run
A row stuck at running usually means a missed webhook. Check the receiver logs for the session id, confirm the signing key, and let the 60s poll backstop or the reaper converge it. The agent_runs row carries provider_session_id and console_url for the cross-reference.
Appendix — enums & schema
reference The frozen contracts. Status and stop-reason live on the agent_runs row; the webhook event kinds and SSE types are the wire vocabulary between the backend, the runner, and the browser.
| Enum | Values |
|---|---|
AgentRunStatus | queued · running · completed · failed · expired · cancelled |
RunStopReason | end_turn · retries_exhausted · requires_action · error · terminated · timeout · invalid_output · cancelled |
AgentWebhookEventKind | run_started · progress · idle · terminated · deleted · updated · unknown |
SSE event_type | session.lifecycle (progress) · run (terminal) |