Agentic runtime
One Run contract, one Agno agent runtime, one deterministic workflow model, and the component boundaries that keep the framework replaceable.
Agentic runtime
The front door and the triggers decide what must run. The runtime decides how to run it.
This page holds the architecture: the components, the boundaries, and the Run contract. Two sibling pages hold the rest.
| Page | Owns |
|---|---|
| Agentic runtime (this page) | Components, boundaries, Run contract, Agno boundary, workflow model |
| Runtime definitions | Draft, validate, publish, snapshot |
| Runtime execution | Run record, Inngest steps, waits, approvals, cancellation, failure, live events |
V1 has one agent runtime: Agno. There is no runtime registry and no backend router.
Component map
Front door / Trigger
│ StartRunCommand
▼
RunManager ──────────► agent.runs
│ dispatch event
▼
Inngest (one function: run.execute)
│
▼
RunExecutor
┌──┴──────────────────┐
▼ ▼
AgentExecutor WorkflowExecutor
│ │
▼ ▼
AgentRuntime WorkflowStepExecutor
│ │
▼ ├──► ToolInvoker
AgnoAgentRuntime └──► child Run (Inngest invoke)
│
└──► ToolInvoker
Both executors reach the world through the same ToolInvoker. Nothing else calls a tool handler.
RunExecutor holds no loop, so it answers no ExecutionOutcome. The Inngest
function owns the segment loop, and each segment step answers its own outcome.
RunExecutor therefore has two jobs and two methods.
class RunExecutor:
async def claim(self, run_id: UUID, organization_id: UUID,
execution_ref: dict) -> Run | None: ...
def executor_for(self, run: Run, *,
steps: WorkflowSteps | None = None) -> SegmentExecutor: ...
claim() answering None ends the function, which is why nothing here returns a
value meaning "do nothing". executor_for() reads run.kind, and both kinds
resolve. There is no second dispatch protocol.
The workflow side is a factory, and the agent side is not. AgentExecutor is
one object for the whole worker. A workflow coordinator cannot be, because
WorkflowSteps closes over ctx.step and ctx exists only inside one function
call. So the cached graph holds what builds a coordinator, and executor_for()
takes the seam of this function run and passes it through. A workflow Run arriving
with no seam names a defect in the caller, because the function reads the kind off
the memoized claim before it creates any step.
The function owns the segment loop. It does not own the node walk. A workflow
is a tree, so a walk in the function would need a durable stack of its own.
WorkflowExecutor holds the recursion, and it creates one durable step per node
through the WorkflowSteps seam. A node answers a NodeOutcome, never an
ExecutionOutcome: only the run ends, so only the run names a next_status.
⚠️ The walk cannot sit inside a step of the function. The SDK refuses a nested
step: it answers STEP_NESTED, which is not retriable. So run.execute reads
the kind off the claim and branches above every step. An agent segment is one step
this function creates; a workflow walk creates its own, between the claim and the
finalize.
Core ownership
| Owner | Responsibility |
|---|---|
RunManager | Run lifecycle, admission, and the start and control boundary. The only writer of status, waiting_on, waiting_ref_id, ended_at, segment_index and resumed_from_wait |
| Inngest | Durable dispatch, retry, wait, timeout, concurrency, cancellation |
RunExecutor | Claim the Run and dispatch it by kind. Once per function run, and nothing else. The workflow side is a factory, because a coordinator closes over the step seam of one call |
AgentExecutor | Re-read the Run, check accrual, subtract the ceilings, build the context brief, call AgentRuntime, and map its result to an ExecutionOutcome. Its seams are AccrualChecker, ContextBuilder and AgentRuntime. The tool contracts are the frozen snapshot, so nothing builds them |
AgentRuntime | Framework neutral agent loop interface |
AgnoAgentRuntime | The V1 implementation of AgentRuntime |
WorkflowExecutor | Walk the node tree of one workflow deterministically, and create one durable step per node |
WorkflowStepExecutor | Execute one workflow node by type, and check accrual and cancellation before it |
WorkflowSteps | The durable step seam. One implementation wraps ctx.step and ctx.group; the test one runs in process |
RunControlRepository | Read and write the agent.run_control cancellation row |
| Tool layer | Every external action and every on demand action |
| Policy | Admission, action and accrual decisions |
| Observability | Spans and usage. SpanRecorder is the one writer of heartbeat_at, which is the single documented exception to the line above |
One Run contract
A Run is the durable product unit. A span is one unit of work inside a Run.
Workflow Run
├─ span: tool, branch, wait, approval, parallel
└─ child Run: an agent or a nested workflow with its own lifecycle
| Unit | Run row | Span |
|---|---|---|
| Top level agent | yes | yes |
| Top level workflow | yes | yes |
| Agent node or subworkflow node | child Run | yes, the child's own run span |
| Tool, wait, branch, approval | no | yes |
parallel container | no | yes |
sequence container | no | no |
The node itself opens no span, and the third row is the child's. None of the eight span kinds names such a node, and the set is closed. The child's run span carries the node ID as its name and hangs from whatever span was current at the node, so the tree reads the same and there is no ninth kind. See observability and operations.
Create a child Run when independent retry, cancellation, ownership or status matters. Do not create a Run row for every small deterministic step.
There is no merge node and no merge span. The node table below holds eight types and none of them is a merge, and observability lists eight span kinds and none of them is either. A parallel node settles when its last branch settles, and the parallel span already covers that join with its own start and end. A sequence writes no span at all, because ordering adds no timing a person can use.
Every Run carries root_run_id. A top level Run points at itself. This is what lets one client subscription follow a whole Run tree.
Product state
status: queued | running | waiting | succeeded | failed | cancelled
waiting_on: approval | event | delay null unless status is waiting
One enum carries the lifecycle, and one field says what a waiting Run waits for. There is no second phase enum. A phase whose values were initializing, executing, waiting_for_approval and finalizing only ever told a person something new inside waiting, and it made every writer keep two fields agreeing.
Inngest attempt IDs, worker identity and queue state stay infrastructure telemetry. A surface may show the attempt count. It must not treat the attempt count as product state.
There is no retrying status. A retry is worker state, and the worker that would write it is the one that died. Writing it on the next attempt means reading the Inngest attempt number, which is the exact leak this rule forbids. Retry stays visible in two honest places: the failed span in the tree, and the attempt correlation field. See runtime execution.
There is no waiting_for_input value in V1. An agent that needs more information ends the Run and states what it needs. The person answers, and the front door starts a new Run.
The Agno boundary
Agno owns:
- the model loop and the reasoning;
- the model interaction;
- the tool selection;
- the session identity that lets its loop continue.
AgencyCore owns:
- the Run state and the Run snapshot;
- authorization, policy and approvals;
- credentials and tool execution;
- context ownership;
- workflow durability;
- idempotency;
- spans and usage.
class AgentRuntime(Protocol):
async def execute(
self,
request: AgentExecutionRequest,
event_sink: AgentEventSink,
) -> AgentExecutionResult: ...
execute() is the whole interface. It returns when the agent finishes, when it needs an approval, or when it reaches a ceiling.
The three contracts of that signature
@dataclass(frozen=True)
class ModelConfig:
provider: str
model: str
temperature: float | None = None
max_output_tokens: int | None = None
@dataclass(frozen=True)
class RunCeilings:
"""Four loop bounds this Run owns, plus the one budget its tree shares."""
max_segments: int # this Run
max_agent_turns: int # this Run, less what earlier segments used
max_tool_calls: int # this Run, less what earlier segments used
max_run_duration: timedelta # this Run, LESS the time already spent
max_cost_cents: int # the ROOT Run's; a child ignores its own
@dataclass(frozen=True)
class SegmentInput:
"""What starts a segment. Agno's arun(input=...) has no default."""
kind: Literal['run_input', 'approval_resolved', 'segment_resumed']
text: str | None = None # original Run input; fallback after a yield
approval_id: UUID | None = None # approval proof; kept across a timed yield
@dataclass(frozen=True)
class AgentExecutionRequest:
run_id: UUID # also the Agno session identity
segment_index: int
principal: Principal
model: ModelConfig # frozen in the snapshot
instructions: str # definition instructions plus the rendered skills
input: SegmentInput # every segment starts on something
context: ContextBrief | None # a fresh brief, or None to reuse the stored one
tools: list[ToolSpec] # model facing contracts, frozen in the snapshot
ceilings: RunCeilings # what is LEFT for this segment, not the Run total
@dataclass(frozen=True)
class AgentExecutionResult:
stop: Literal['completed', 'needs_approval', 'ceiling', 'cancelled', 'failed']
result: RunResult | None = None # stop = completed
approval_id: UUID | None = None # stop = needs_approval
approval_expires_at: datetime | None = None
ceiling: str | None = None # stop = ceiling; the field that was reached
error: RunError | None = None # stop = failed
turns_used: int = 0
tool_calls_used: int = 0
approval_resolved carries no text, because the segment is a continue. A segment
after approval does not start the loop again. The runtime rehydrates the paused run, resolves
the requirement the approval names, and continues from there, so the approved outcome
fills that call's own result slot. See
tools and integrations
for the measurement behind that.
segment_resumed carries the original Run input as a fallback, not as the
usual next prompt. The wall clock can interrupt the session write at any
point. The Agno runtime continues a paused run when the stored session holds
one. With saved history and no pause, it starts one stable continuation turn
with Continue the task from the saved conversation. With no saved session,
it restarts from text. The runtime alone reads this framework state. A timed
approval continuation can also carry approval_id. The runtime applies it
only when it continues the saved paused run. It drops the proof before it
starts a fresh turn from saved history or text.
Every field is an AgencyCore concept. No Agno type appears, which is the whole point of the protocol.
stop is a closed set, and it is the only thing AgentExecutor branches on. An implementation that cannot say why it stopped cannot satisfy this contract.
There is no session_id field, because the Run is the session. The runtime passes str(run_id) to Agno, and agent.sessions is keyed on the Run. A second identifier would be a second unique key that can disagree with the first.
Every fresh framework run carries text, because the framework demands it.Agent.arun(input=...) has no default on Agno 2.5.10. Segment 0 carries the
Run input. A segment after a wall-clock yield carries either the stable
continuation prompt or the original Run input. A segment after an approval
continues the paused framework run instead of starting a fresh one.
cancelled is a stop reason, because the adapter is the code that sits at the safe boundary. Cancellation is read before and after a tool call, and inside a segment only the tool adapter reaches that point. Without this member the adapter would have to report a cancel as a failure, and RunManager would then write failed over a Run a person stopped.
The event sink
class AgentEventSink(Protocol):
"""The one way the agent loop reports progress. Spans come from SpanRecorder."""
async def text_delta(self, text: str) -> None: ...
async def tool_proposed(self, tool_name: str, arguments_preview: str) -> None: ...
Two methods, and they map to the two live events a span cannot produce: text.delta and tool.updated. Everything else on the wire is a projection of the span lifecycle, so it belongs to SpanRecorder. See observability and operations.
The sink is best effort. It never blocks the loop, and it never raises into it. A dropped delta costs a repaint, and the durable record is unaffected.
The sink is wrapped, because an exception into the loop is invisible. Agno turns any exception raised inside a tool call into a tool result and continues, so a sink that raises does not fail the segment. It teaches the model something false instead. Every sink call sits inside a try.
What the runtime owes besides the loop
The event sink is not the whole duty. Three durable facts about a model turn have no other writer, because the model call happens inside Agno.
| Fact | Written by | What breaks without it |
|---|---|---|
One llm span per model turn | AgnoAgentRuntime | max_agent_turns reads the span tree, so the turn ceiling never fires |
One ai_usage_log row per model call | AgnoAgentRuntime | max_cost_cents reads the meter, so the money ceiling never fires. This is cross document invariant 13 |
heartbeat_at, through the span open | SpanRecorder | A segment whose model turn runs longer than stale_after is failed by the reaper while it is healthy |
The span opens when the turn starts, not when the segment ends. A span written after the fact proves nothing about a worker that died, and it writes no heartbeat while the turn is running. So the segment runs in streaming mode and the bridge maps the Agno event stream onto both duties: ModelRequestStarted opens the llm span, ModelRequestCompleted closes it and writes the usage row, and RunContent feeds text_delta. One mechanism serves the sink, the spans, the meter and the heartbeat.
Every call into the framework streams, and the continue streams too. A
segment reaches the model through arun() and through one acontinue_run() per
drained pause. Stream the first and not the second and the heartbeat stops at the
first write tool, which is exactly the long segment the reaper is watching for.
Both take stream=True, stream_events=True and yield_run_output=True; the
last one is what returns the RunOutput at the end of the event stream, and
without it the runtime has the events and not the pause.
The span pair spans two events, so the bridge holds it open. span() is a
context manager and the open and the close arrive on two iterations of one event
loop. The bridge keeps the handle in an AsyncExitStack between them. It does
not open the span at the close and read the timestamps back.
ModelRequestCompleted carries tokens and no price and no duration. The
runtime prices the call from the one pricer, and it times the turn itself. See
observability and operations.
SpanRecorder needs no new parameter. Span identity is ambient, read through current_run_id(), current_root_run_id() and current_span_id(). See observability and operations.
Agno bounds tool calls and does not bound turns. Agent(tool_call_limit=...) exists on 2.5.10 and no turn limit does, so max_agent_turns is enforced by the runtime's own counter over the llm spans it opens. tool_call_limit is not used at all: a replayed tool call counts nothing everywhere else in the platform, and it would still spend that limit. One ceiling gets one mechanism.
What ends the loop early, and how
Counting a ceiling is not stopping one. The framework gives the runtime two
places to act, and every early ending below uses one of them. Nothing abandons
the event iterator: an abandoned iterator leaves an llm span open, writes no
session, and leaves the framework holding a run nobody finishes.
| Point | Reached by | How it leaves |
|---|---|---|
| The adapter, before a read tool call | every read tool | it records the segment stop, then raises StopAgentRun |
| The drain loop, before every paused call | every write and send tool | execute() returns |
A ceiling stops at the adapter. A model turn only follows a tool call, so
one check before every call bounds both max_agent_turns and max_tool_calls.
A paused write call never reaches the adapter, so the drain loop checks the same
two remainders itself.
⚠️ The drain loop checks before every call, not once per pause. One turn can hold ten write proposals. Checked once at the top of the pause, a remainder of one sends ten emails and the ceiling stops only the next pass. This is the side effecting path, so it is the one that must not overshoot.
The segment stop is read before is_paused. One turn can hold a read tool
that failed and a write tool that paused. RunOutput.status then reads PAUSED
while the adapter already recorded failed. Read the pause first and the runtime
drains a segment that is already over.
A cancel in the drain loop is a stop and not a fault. ToolInvoker raises
CancelRequested at the same boundary whether the call paused or not, and
outside the loop that is an ordinary raise. The runtime maps it to
stop = cancelled. Left to propagate, it reaches Inngest as a failure and
RunManager writes failed over a run a person stopped.
A model fault is swallowed too, and the type is what decides the retry.
Agno catches every exception a model call raises. Measured on 2.5.10 with a
scripted model: a ModelProviderError, a RetryableModelProviderError and a
plain RuntimeError each end the stream with a RunErrorEvent carrying a
string, and no RunOutput follows. By then the type is gone, and the three
need three different answers.
So the runtime wraps the model's two async entry points for the segment, keeps
the first exception in a ModelFault beside the segment stop, and classifies
once the stream has ended.
⚠️ The type names do not answer the retry question, and reading them that way
inverts it three ways. Measured on 2.5.10: ModelRateLimitErrorsubclasses ModelProviderError, so a rule written against the parent
catches every 429 and ends a run one replay would have saved.
ModelAuthenticationError does not subclass it, and OpenAIChat re-raises
that one un-wrapped, so the same rule misses a 401 and replays a bad key for
ever. RetryableModelProviderError is not an AgnoError at all, and only the
Gemini class raises it, so a rule written against it answers no for every
provider this platform builds. Model.retries defaults to 0, so the framework
retries nothing on its own either.
AgnoError is the base all of them share except the last, and it always carries
status_code. That status is the one rule.
| What was raised | What the runtime does |
|---|---|
AgnoError, status 4xx other than 429 | stop = failed, retryable = False; the vendor answers the same way every time |
AgnoError, any other status | re-raise it; Inngest replays the step |
RetryableModelProviderError | re-raise it; it is not an AgnoError, so the rule above never sees it |
| anything else | re-raise it; a platform fault must reach Inngest |
| nothing recorded | stop = failed, runtime_contract |
A malformed body, a bad key and a model that does not exist are all 4xx, and all
three are refused identically on every attempt. A rate limit is the one 4xx that
is not, and ModelRateLimitError carries 429 by default, so the named clause in
the rule is what replays it. That clause is not dead code: drop it and every
rate limit becomes terminal.
⚠️ The fault is cleared before every framework call. A segment that drains two pauses makes three, and the framework recovers from a retryable fault on its own. Kept across those calls, a rate limit the first call recovered from decides the answer for a refusal on the third, and Inngest replays a segment that fails identically every time. The mirror is worse: a recovered refusal makes a genuine rate limit terminal, and a run one replay would have saved ends for good.
Without the wrapper a rate limit ends the run instead of replaying the step, and a database outage during a model call does the same.
The turn is the fault window, and the framework call opens one too. The
bridge clears the fault at every ModelRequestStarted, because one framework
call holds many turns and the framework recovers from a retryable fault in
place. A call that fails before it reaches a turn emits no start event, so the
runtime clears it at the call boundary as well.
A turn that failed closes its span as an error. The bridge raises inside the
span's own block rather than after it. Recorded and raised afterwards, the block
would exit with no exception, the recorder would write ok with no error, and a
run that ended on a refused request would show every model turn green with no
usage row and nothing to read.
A segment after an approval continues the paused run and never starts the
loop. The runtime finds the paused run in the stored session and enters the
drain loop on it. A fresh arun pays for every earlier turn a second time, and
the approved call's own empty result slot stays empty, so the effect a person
allowed never reaches the model. A session that holds no paused run ends the
run on paused_run_absent, which no retry can fix.
A segment after a wall-clock yield is defensive. A saved paused run also
continues through acontinue_run. If the yield followed an approval, that
saved pause gets the approval proof. A saved session with no pause starts a new
arun with one stable continuation prompt and no old proof. No saved session
restarts the original Run input, also with no old proof. A matching completed
tool journal entry returns its stored result; an interrupted entry keeps the
journal's existing lease behavior.
A completed segment always carries a RunResult. AgentExecutor answers
runtime_contract and fails the run when completed arrives with no result, so
a model that answered an empty string would fail a healthy run. The runtime
builds the result from what it holds and never returns None there.
Three result types, and why each exists
They are easy to confuse, so name the question each one answers.
| Type | Answers | Lives |
|---|---|---|
AgentExecutionResult | Why did this segment stop? | inside the agent runtime boundary |
ExecutionOutcome | What must RunManager do next? | between any executor and the manager |
RunResult | What did the run produce? | on the run row, durable |
A workflow produces no AgentExecutionResult, and both executors produce an ExecutionOutcome. That is why the middle one is not redundant.
@dataclass(frozen=True)
class ExecutionOutcome:
next_status: Literal['running', 'succeeded', 'waiting', 'failed', 'cancelled']
result: RunResult | None = None
error: RunError | None = None
waiting_on: WaitingOn | None = None # set when next_status is waiting
waiting_ref_id: UUID | None = None
waiting_expires_at: datetime | None = None # the Inngest wait timeout
The wait carries its own deadline, because nothing else on this side holds
it. The Inngest wait computes timeout from the approval expiry, and the
segment that raised the approval is the only code that read it.
AgentExecutionResult.approval_expires_at holds it inside the agent boundary,
and AgentExecutor answers an ExecutionOutcome, so a type without this field
drops the value at exactly the boundary the function needs it. The alternative
is a second query over agent.approvals for a fact the raiser already held,
which is the same trade the agent boundary already refused one layer down.
next_status says what the durable function does next. It is a subset of
RunStatus. queued is absent because start() owns it. running is the
segment wall-clock yield: it asks the loop to advance and start another
segment, and it asks RunManager to write nothing.
Two values name lifecycle writes, and three do not. Reading every value as
a RunManager instruction is how code writes a status another boundary owns.
next_status | RunManager call | Why |
|---|---|---|
running | none | the segment loop advances and continues |
succeeded | succeed(run_id, result) | the executor is the only writer |
failed | fail(run_id, error) | the executor is the only writer |
waiting | none, or an idempotent re-assert | mark_waiting() already ran inside the segment, before it returned |
cancelled | none | cancel() already wrote the status |
cancel() takes an ActorIdentity the executor does not hold, so an implementer who reaches for it has to invent an actor, and it would overwrite cancel_requested_by with a machine where a person belongs. waiting is the subtler of the two: the approval row, mark_waiting() and the wait all happen before the segment step returns, so the outcome reports a transition that is already durable. Re-asserting it is legal, because waiting -> waiting is a legal cell, but it writes nothing new.
claim() returning None produces no ExecutionOutcome at all. A Run
cancelled between the dispatch and the claim is no longer claimable, and there
is nothing for RunManager to be told. RunExecutor returns and the Inngest
function ends. running is not this absence: it is an explicit instruction to
advance the bounded segment loop.
Source ExecutionOutcome.next_status
outer segment timer -> running
runtime: completed -> succeeded
runtime: ceiling -> succeeded, with RunResult.partial_reason
runtime: needs_approval -> waiting, waiting_on = approval
runtime: cancelled -> cancelled
runtime: failed -> failed
The outer timer does not add an AgentExecutionResult.stop value. It wraps the
agent executor at the durable step boundary and produces running directly.
There is no cancel() method on this protocol. Cancellation is cooperative and durable, and the executor reads the agent.run_control row at safe boundaries. An in memory handle does not survive a worker restart, so it cannot be the control boundary. Steering is deferred to V2, so nothing on this protocol carries it either. See runtime execution.
AgentExecutionRequest carries AgencyCore concepts only: the Run ID, the model configuration, the instructions, the authorized ContextBrief, the rendered skills, the model facing tool contracts, the Principal and the ceilings.
If a second runtime is ever added, it must satisfy this contract. Do not build the selection machinery before that need exists.
Where the session lives
An agent Run can span several segments, and a segment can be separated from the next one by a three day approval. The next segment continues the same Agno session, so the message history must survive a worker that no longer exists.
That store is real, and it needs a name. It is agent.sessions, one row per Run, and runtime execution owns its schema. Do not restate the columns here.
| Concern | Owner |
|---|---|
| Write | AgnoAgentRuntime, at the end of every segment that produced an outcome |
| Read | the next segment, before it continues the loop |
| Delete | Run retention, with the Run it belongs to |
A segment that will be retried writes nothing. A retryable failure leaves a half finished turn in the session, and Inngest replays the whole step. Writing it makes that wreckage the baseline the retry rehydrates. Not writing it leaves the clean pre-segment session in place, which is what a replay is for. Runtime execution lists the exit paths and which of them write.
Two consequences follow, and both are already relied on elsewhere.
- The earlier
ContextBriefis still in that history. That is why a rebuilt brief replaces the system block instead of appending. See state and knowledge. - It is Agno's state, not ours, so nothing above
AgnoAgentRuntimereads it. The rejected alternative was rebuilding the history from spans. Spans are bounded and redacted, so they cannot reconstruct a model conversation.
The stored row carries the instructions too
Replacement is native, and it has a cost that is easy to miss. Agno rebuilds the system message from instructions on every run, and it never replays the stored one. So the system block is not a thing the session keeps. It is a thing the caller supplies again each time.
AgentExecutor sends context = None on an ordinary later segment, meaning "the world did not move". If the runtime then passed instructions without a brief, the brief would simply be gone from segment 1 onward. Nothing would fail. The agent would forget the rows it was shown, and the run would look healthy.
So the runtime stores what it composed, beside the state it stored.
agent.sessions.agno_state = {
'agno': <AgentSession.to_dict()>, # Agno owns this half
'instructions': '<the composed system block>',
}
context = None therefore means reuse the stored instructions, and the column is a runtime owned envelope rather than a raw Agno dictionary. agent.sessions needs no second column and no migration, and rule 2 below is unchanged: nothing above AgnoAgentRuntime opens either half.
The history is a setting, and it has a floor
Rehydrating the session is not the whole answer. Agno replays a stored history
into the model only when the agent asks for it, so the runtime builds every
agent with add_history_to_context=True. Leave it at its default and a resumed
segment reads a session it never uses, and the model answers a question it was
already told the answer to.
num_history_runs bounds how many earlier runs come back. One segment is one
Agno run, so the bound is a segment count and it must be at least
max_segments. Set it lower and a long run silently loses its oldest segments
while every ceiling still reads healthy.
Workflow model
A workflow is deterministic coordination. It is not a second reasoning hop.
Eight node types. Two of them are containers.
| Node | Kind | Behavior |
|---|---|---|
sequence | container | Run the children in order |
parallel | container | Run a fixed set of children at the same time |
tool | step | Call one tool through ToolInvoker |
agent | step | Run one agent as a child Run |
subworkflow | step | Run one workflow as a child Run |
branch | step | Pick one child with the shared ConditionEvaluator. It declares a default |
wait | step | Hold for an event or for a delay |
approval | step | Hold for a person |
A workflow has no loops, no expression language, no private schedule and no model planning between steps.
Where runtime fan out lives
The node set of a workflow is static. The workload of a node is not.
A step may process many items that only exist at run time. That fan out lives inside one tool step or one agent step, and the step reports one result. Signals Search uses this: one step searches for people across the companies that survived pruning.
This keeps the workflow shape checkable at publish time. It also keeps bounded fan out possible. Do not add a dynamic fan out node type to get it.
Skills
A skill is reusable guidance. The runtime renders it into the agent instructions.
- Prefer a skill when the responsibility stays the same and the procedure is reusable.
- The tool IDs of a skill must be a subset of the tool IDs of the agent.
- A skill cannot grant a capability.
- A skill is not a workflow node type.
- V1 stores short procedure text in Postgres. There are no scripts, no sandbox and no Agno filesystem skill packages.
The runtime renders skills once, when it freezes the Run snapshot. Execution reads the rendered text. It does not render again.
Code layout
runtime/
runs/
models.py Run, RunSnapshot, RunResult, RunStatus, WaitingOn,
StartRunCommand, StartRunResult, ExecutionOutcome
repository.py RunRepository: one conditional transition primitive
transitions.py the legal from-states for every lifecycle method
manager.py RunManager
admission.py AdmissionGate: one policy method either side of the insert
control.py RunControlRepository: the cancellation row
start_key.py the start key of a run
dispatch.py the Inngest carrier for a dispatch and a cancel
definitions/ draft, validate, publish, snapshot. The definitions page
owns this package
execution/
executor.py RunExecutor
agent.py AgentExecutor
workflow.py WorkflowExecutor, the deterministic node walk
workflow_step.py WorkflowStepExecutor, NodeOutcome, the child start
tool_node.py what a workflow `tool` node hands the tool layer
child_actor.py the actor a node starts its child run with
definitions.py the run time reader of the subworkflow targets
steps.py WorkflowSteps, the durable step seam
outcome.py ExecutionOutcome -> RunManager calls
agent/
protocol.py AgentRuntime, AgentExecutionRequest/Result/SegmentInput,
AgentEventSink. RunCeilings is NOT here: three packages
read it, so it lives at shared/ceilings.py beside bound()
agno/
runtime.py AgnoAgentRuntime
segment.py the mutable state of one segment
adapter.py AgnoToolAdapter: the declaration and the stop signal
bridge.py Agno event and result normalization
session.py agent.sessions: the envelope read and the envelope write
events/
publisher.py RedisRunEventPublisher, the channel scheme and the mirror
agent_sink.py the agent events a span cannot produce
inngest/
wiring.py build_platform(), the composition root of the worker
execute.py run.execute, the two lanes and the segment loop
failure.py run.failed, the inngest/function.failed backstop
reaper.py run.reaper, the cron and the two orphan sweeps
idempotency_sweeper.py idempotency.sweeper, the cron that forgets a claim
retention_sweeper.py retention.sweeper, the cron and the three windows
The node models are not in execution/. Both definitions/ and execution/
read them, so they sit beside RunCeilings in the package that imports neither.
shared/
ceilings.py RunCeilings, the five bounds one run may spend
bounds.py bound(), the one payload boundary
workflow_nodes.py the eight node types, and the reader of the tree
workflow_graph.py the reference graph: one cycle guard, and one depth
workflow_references.py what a node may read, and how it reads it
conditions.py ConditionEvaluator, the one condition language
identity.py Principal and ActorIdentity
refs.py a pointer to a row, where a payload carries no body
database.py the PostgREST transport facts: SERVER_NOW, the
row cap, the filter batch size
retry.py retried(), the one transient retry both planes share
ttl_cache.py the bounded per-process cache the policy reads share
LiveEventPublisher is declared in governance/observability/recorder.py,
beside the caller that writes the span half. Phase 1 ships the Protocol and a
default that publishes nothing, and a recorder built with None still
publishes nothing today.
Phase 2 added the one implementation, RedisRunEventPublisher, in the
runtime/events/ package above. Three classes call it. SpanRecorder writes
the span half. PublishingAgentEventSink writes the two events a span cannot
produce, and runtime/agent/agno/bridge.py reports them to it through the
AgentEventSink port AgentExecutor owns. RunManager writes the run
lifecycle events.
Only runtime/agent/agno/ imports Agno. Everything above it works on the neutral contracts.
RunCeilings lives in shared/, and the reason is a package cycle. The
agent protocol carries it, AccrualChecker takes it, and ToolInvoker checks a
metered call against it. Declare it inside runtime/agent/ and governance
imports runtime, while RunManager already imports governance for
PolicyDecision. Two forbidden contracts pin the direction: nothing under
services/ or governance/ imports runtime/, and shared/ imports none
of the three.
AgnoToolAdapter lives here and not in the tool package. It declares an Agno callable and it raises an Agno exception, so it is framework code. The tool page already says the adapter is the piece replaced when the framework changes, while the registry, the invoker, the policy and the handlers do not move. Putting it under services/tools/ would put Agno in two packages, and the import contract could then no longer name one.
Rules
- One Agno agent runtime in V1.
- No backend selector, no runtime registry, no capability matrix.
RunManagerowns the product lifecycle. Inngest owns the durability.RunExecutorclaims and dispatches, once per function run. The per-segment and per-node work belongs toAgentExecutorandWorkflowStepExecutor.AgentExecutormapsAgentExecutionResulttoExecutionOutcome. It never returns the first where the second is declared.- A segment re-reads the Run row before it does anything else. The claimed Run is one snapshot of two columns that move.
- A status other than
runningends the segment.agent.run_controlis the boundary inside a segment, not between two. - A stop before the first model call fails the Run. The same stop after one succeeds with a
partial_reason. - A retryable platform fault leaves the executor as an exception. Only a terminal failure becomes an
ExecutionOutcome. RunExecutoranswers a claimed Run and an executor. It answers noExecutionOutcome, because it holds no loop.succeededandfailedname aRunManagermethod.waitingandcancelledare reports of a write that already happened.runningis a segment yield. It advances the durable loop and writes no Run status.- A per-Run ceiling counts spans of that Run. Only the cost ceiling sums the tree.
- Agno types stay inside
runtime/agent/agno/, which holds the runtime, the tool adapter and the bridge. - The agent session is durable, scoped to one Run, and read only by
AgnoAgentRuntime. - The session row carries the composed instructions, because Agno rebuilds the system block from them on every run.
- A segment that will be retried writes no session.
- The stop signal lives on the segment, never on the runtime instance.
- The runtime opens one
llmspan per model turn and writes one usage row per model call. - The loop is left at the adapter or at the drain loop. Nothing abandons the event iterator.
- The segment stop is read before
is_paused, because one turn can hold both. - One pricer answers the cost of a model call, and the non-cached input count is provider specific.
- Every call into the framework streams, including each continue.
- One ceiling gets one mechanism. The runtime counts; the framework's own limit is unused.
- Every agent action goes through
ToolInvoker. - Not every workflow step becomes a Run.
- The node set of a workflow is static. Fan out lives inside a step.
WorkflowExecutorowns the node walk. The Inngest function owns the segment loop only.- The function branches on
run.kindabove every step. A workflow walk creates its own steps, and a nested step is refused with a non-retriable error. run.executeanswers the status, the result and the error. A parent node reads that mapping to settle its child node, so the status alone empties everysteps.<id>.outputin the tree.- A workflow
toolnode counts againstmax_tool_callsby thetoolspans of its own Run, exactly as an agent segment subtracts them. - One node body is one durable step, and every write a node makes sits inside one. The SDK re-executes the function body once per step, so a write outside a step happens again on every pass.
- A
parallelnode runs its branches through the parallel primitive of the SDK, in race mode. A plain loop over the branches serialises the node in silence. - A business failure inside a node is a returned value. Only a retryable platform fault raises.
- Workflow nesting is validated and capped before publish, and the same function re-checks it when a subworkflow node starts a child Run.
- Every Inngest function runs on the Connect worker. The web process serves no Inngest function, whatever the function costs.
Minimum contract tests
RunManager.start()is idempotent for the same command key.- A terminal Run stays terminal, whatever transition arrives afterwards.
- An edit to a published definition after start does not change the Run snapshot.
- A policy change does reach the next checkpoint of an in flight Run.
- The
AgentRuntimeconformance suite normalizes output and events without Agno types. - A second segment still carries the brief, even when
contextis None. - Two segments running at once in one process do not read each other's stop signal.
- A model turn opens an
llmspan while it runs, so a long turn keeps the heartbeat fresh. - A turn ceiling reached inside a segment stops the loop at the adapter, and the segment ends
ceiling. - A read tool that fails beside a write tool that pauses ends the segment
failed, not paused. - A cancel raised in the drain loop ends the segment
cancelled, and raises nothing at Inngest. - A request the vendor refuses fails the run once, and Inngest replays no step for it.
- A rate limit and a connection fault both replay, whatever the exception class is called.
- A model turn that failed leaves a span whose status is
error. - A tool contract the snapshot froze and the runtime cannot declare fails the run once.
- A rate limit and a platform fault inside a model call both leave the runtime as exceptions, so Inngest replays the step.
- A segment after an approval continues the paused run, and the framework makes no second first turn.
- A segment after a wall-clock yield handles a saved pause, saved history, and no saved session.
- A timed approval continuation keeps its proof for a saved pause and drops it before a fresh turn.
- A model call priced from an OpenAI response prices its cache reads one time.
- A completed segment whose model answered nothing still carries a
RunResult. - A platform fault inside a tool ends the segment, and the model claims nothing.
- A paused write call is decided by policy before it runs, and an allowed one never reaches Inngest.
- A segment after an approval continues the paused run; it never starts the loop again.
- A workflow tool node uses
ToolInvoker. An agent node and a subworkflow node use child Runs. - A workflow node walk is deterministic, so a replay produces the same node order and the same step IDs.
- A
parallelnode of two branches runs both at once, and a branch that parks does not hold its siblings. - A failing branch settles its siblings rather than cancelling them, and the node body raises nothing for it.
- A retried child node meets
duplicateand the tree ends with one child. - A
toolnode produces onetoolspan, written byToolInvoker, and no second one. - A workflow run creates its node steps at the body level of the function, and never inside a segment step.
- A successful
agentnode reads its child's output intosteps.<node id>.output, and a failed one reads the child's own error code. - A fan out of
toolnodes stops atmax_tool_calls, and the calls already in flight are the only overshoot. - A
subworkflownode re-checks the depth cap against the live published definitions, not against the parent's frozen snapshot. - Every Run in a tree carries the same
root_run_id. - Cancellation is safe and idempotent.
- A Run of several segments spends its turn ceiling once, not once per segment.
- A workflow of ten agent children gives each child its own turn and tool ceilings and one shared cost ceiling, and the tenth is not handed zero.
AgentExecutor.execute()returns anExecutionOutcomefor every stop reason.- A Run cancelled between dispatch and claim ends the function with no
ExecutionOutcomeand no further write. - An outcome of
waitingcarries the deadline the Inngest wait times out on, and the function reads no approval row. - An outcome of
waitingwrites no new status, because the segment already did. - Accrual is checked at the top of every segment, on a Run whose
RunExecutorreturned long before. - A segment resumed by a fresh worker reads the same agent session.
- A segment reads the Run row itself, so segment 4 sees
segment_index = 4and not the value the claim answered. - A Run cancelled between two segments ends
cancelledbefore the runtime is called. - A segment that follows a resolved approval receives the approval id, on a Run whose
waiting_ref_idis already null. - A timed segment returns
running, advances once, and starts the next segment without a wait. - A nested
TimeoutErroris not a segment yield unless the outer timer expired. - A yield on the last allowed segment keeps the existing
limit_reachedending. - A stop before the first model call fails the Run. The same stop after one succeeds with a
partial_reason. - A retryable platform fault leaves
execute()as an exception, and produces noExecutionOutcome. - The tool call remainder ignores a span marked
replayed.