Runtime definitions
Editable drafts, one published configuration per definition, template forks, deterministic validation, and the Run snapshot that keeps in flight work stable.
Runtime definitions
A definition is the configuration the runtime may execute. A draft is its editable copy.
Draft -> validate -> publish -> the current runnable definition
Publishing is a safety boundary. It is not a revision system and not a history system.
One lifecycle
agent.definitions
id
organization_id
kind: agent | workflow | skill
name
origin: platform | custom
source_definition_id
state: draft | active | disabled
draft_config
published_config
published_at
updated_at
| State | Meaning |
|---|---|
draft | Never published. It cannot run. |
active | It has a published_config. New Runs use it. |
disabled | It has a published_config and it is switched off. New Run trees are refused. |
draft --publish--> active
active --disable--> disabled
disabled --enable--> active
disabled --publish--> active a publish is a deliberate act, so it also enables
draft --delete--> gone a draft only
disabled is reversible, and enable revalidates. A definition this one references may have been disabled while this one was off, so enable runs the same validation a publish runs. Without that, an enable can return a broken definition to service.
⚠️ enable validates the stored published_config, derived scope keys included. The stale scope pair is the break it most needs to find, because the publish that caused it skipped this definition: referrer revalidation reads active referrers only, and this one was switched off. The comparison is between a stored pair and a recomputed one, so validating a config with the pair stripped skips the check in silence and returns the definition to service with a required_scopes policy admission then trusts.
A draft is the only row that deletes. A published row is disabled, never deleted: runs_definition_fk takes ON DELETE RESTRICT, and a Run's audit trail must not be deletable from under it. delete_draft therefore refuses any row whose state is not draft. Conversational authoring creates a draft per attempt, so without this the abandoned ones accumulate with nothing to clear them.
A definition stays active while an admin edits its draft. The UI shows unpublished work by comparing draft_config with published_config. There is no fourth state for it.
New Runs use published_config. Runs already in flight keep their frozen snapshot.
Disable stops a new tree, and not a tree already running
disable refuses a new Run tree. It does not refuse a child of a tree that was admitted while the definition was still active. A workflow can wait three days for a person and then start its next agent node; refusing that child would fail a Run an admin never asked to stop, and POST /runs/{id}/cancel is the control that stops one.
assert_run_shape() already holds this: it raises for a disabled definition only when parent_run_id IS NULL.
RunManager must hold the same rule, and today it does not. definition_not_published is documented for a draft or disabled definition on every start, and a workflow node starts a child through the same RunManager.start(). RunManager runs before the trigger, so its check wins and the trigger's carve-out never fires. The result is a workflow that fails mid Run on a disable that was meant to stop new work only.
parent_run_id IS NULL draft -> refuse disabled -> refuse
parent_run_id IS NOT NULL draft -> refuse disabled -> allow
A draft is refused on both sides. A draft has no published_config, so there is nothing to run whatever started it. This is the same asymmetry the schema encodes, and the two must agree. See runtime execution.
A definition cannot return to draft once a Run references it. A draft holds no published_config, so a draft with a live Run is unmanageable: ON DELETE RESTRICT refuses the delete, and the draft rule refuses every new Run. A trigger refuses the reversion instead. It locks the definition row while it reads, because this rule and the draft rule guard one invariant from opposite sides. Without the lock neither sees the other's uncommitted work, and both commit.
Product capability bindings
Phase 7 adds optional product metadata to tenant definitions. It does not add a fourth definition kind. The capability contract defines the five IDs and their public schemas.
A binding belongs to one definition row. Its executor UUID is that row's id, never a second editable pointer.
Persist nullable capability_binding metadata and a separate capability_active Boolean on agent.definitions.
The closed metadata object holds capability_id, contract_version, name, description, input_schema, output_schema and platform_managed.
Persist the derived required scopes with the published workflow configuration. The registry reads these values from one revision.
Custom definitions without product metadata keep their existing lifecycle and Front Door path.
Platform templates have no active tenant binding and cannot execute for an organization.
Enforce one active (organization_id, capability_id) binding in storage, including concurrent installation and replacement.
A binding's active state is separate from the definition's draft, active or disabled state.
This permits an older executor to remain valid for published parent references while a newer executor serves direct product starts.
An active binding must name an active, published Workflow in the same organization.
A draft schema, missing executor, disabled executor, stale scope set or duplicate binding is unavailable.
Do not infer a binding from a display name, provider ID or mutable draft configuration.
Binding storage and lifecycle
ENG-2287 owns these storage and Python persistence rules:
- An unbound row has SQL
NULLmetadata andcapability_active = false. Migration does not infer or install bindings. - Metadata accepts only the five stable IDs, a positive integer version, bounded product text and valid closed root object schemas.
- Metadata is at most 65,536 UTF-8 bytes. Each schema is at most 32,768 bytes. Schema references must be local.
- Attach metadata only to an active, published tenant Workflow. Validate the stored graph and its derived scopes first.
- Attach with the tenant, unbound state and exact
updated_attoken. A stale token writes nothing. - The first attachment cannot change the published configuration or dependency references in the same write.
- Once attached, metadata, definition identity, ownership, published configuration and dependency references cannot change in place.
- Draft edits remain separate. Generic republish refuses a bound definition. A normal fork creates an unbound draft.
- Activation uses the tenant and exact
updated_attoken. A partial unique index permits one active binding per tenant and stable ID. - A concurrent activation conflict returns a bounded conflict result. The caller reloads before retrying.
- Disable clears activation in the same database write. Enable validates the executor but leaves its binding inactive.
- Deactivation keeps the executor active for published parents. Bound rows cannot be deleted, including inactive versions.
- Required scopes remain in
published_config. Python reads them withrun.start; missing or malformed scope data fails closed. - Authenticated users can read binding metadata in their tenant through existing RLS. Only the service role can write it.
Binding immutability protects the owning definition row. It does not freeze unbound descendants. ENG-2303 owns managed descendant revision protection. ENG-2298 owns capability metadata in the frozen Run tree. Their exit tests must cover a child instruction or Skill change that preserves the same scopes.
Registry reads
ENG-2275 owns these read rules. runtime/definitions/registry.py holds CapabilityRegistry, beside the repository it reads.
It sits in runtime, so entry_control and surfaces may both read it and services may not.
DefinitionRepository.list_capability_bindingsreads the bound rows of one organization. The active rows come first.- It reads the bound rows and not the active ones alone, so a disabled executor does not read as a missing installation.
- The projection reads the two scope arrays by JSON path. A catalogue read never carries a published workflow graph.
- One row maps to one
CapabilityRow.row_to_definitionraises on an invalid binding, and one such row must not fail the whole read. - The registry iterates the five stable IDs and never the rows. A stored ID outside that vocabulary names no capability.
- Two active bindings refuse. The registry picks no executor by row order.
- The registry holds no cache and no page. There are five IDs, and a disable must take effect at once.
Initial install and reconciliation
ENG-2277 owns the initial installer. One code manifest describes every managed definition and the five V1 bindings. Each manifest node has a stable key, revision, kind, name and configuration factory. Each binding has the exact product text and schemas for contract version 1.
The installer derives each definition UUID with UUIDv5 from the organization, stable key and revision. Each released node revision is immutable. A change to a node kind, name or configuration needs a new revision of that node, and ENG-2303 owns the switch that activates it. The manifest keeps every released revision of every node, oldest first, so a superseded row stays identifiable as a managed row.
Two tests hold the table honest. A released (key, revision) pair freezes the node kind, name and
configuration. A node revision is never lower than the revision of a node it references, so a child bump
propagates up to the bound root. That propagation is not a style rule: the binding trigger freezes the
published configuration of a bound row, so a parent cannot re-point at a new child in place.
Golden tests freeze the full manifest and every derived UUID. The database primary key makes concurrent creation idempotent. The manifest and deterministic IDs record managed definitions without another table. Names remain display text and never identify managed rows.
The installer uses a platform-only definition writer. It takes an organization ID and cannot be reached from tenant authoring surfaces. It confirms that the organization exists before its first write. It reuses the existing definition validator and publish rules. It never impersonates an organization user or trigger.
Reconciliation processes manifest nodes in dependency order:
- Read the deterministic row before a write.
- Create a missing row as a draft.
- Publish an exact draft through the normal validator.
- Reuse an exact published row.
- Attach the exact binding to the root workflow.
- Activate the root only after the complete tree validates.
An existing row must match its manifest kind, name and resolved configuration. For a published row, its stored references must also match the references derived from that configuration. A binding must exactly match the manifest version, text, schemas and managed marker. A mismatch is a conflict. The installer never changes the conflicting row. It retains exact managed nodes that it created before it found the conflict. This rule also protects a managed draft that an organization admin edited.
The installer never locates a row by name.
It reaches a row by its deterministic UUID and compares the whole binding, so it never changes a tenant binding.
platform_managed is a field of that compared binding; no code reads it on its own.
Any tenant binding for the same capability produces conflict, whether it is active or inactive.
The installer then attaches or activates no managed binding for that capability.
It never replaces an active binding during initial installation.
It may activate an exact, inactive managed binding when no active binding exists.
It reports a disabled executor and does not enable it.
It reports invalid or duplicate bindings and does not select one by row order.
Each capability returns one bounded result: current, installed, missing, disabled, conflict, invalid or blocked.
Infrastructure faults raise for retry. Stable tenant or data conflicts return a report and do not retry without a state change.
An organization that no longer exists returns the run-level result absent_organization and no capability results.
One durable reconciliation function owns an organization run.
Its event trigger handles new organizations and operator backfills.
A scheduled sweep pages all organization IDs and sends one reconcile event for each organization.
Each sweep run handles one bounded page.
A cursor continuation event starts the next page.
Thus, the step count for one run stays bounded, regardless of the organization count.
Complete installations return current without a write.
The database rules provide idempotency after Inngest's event deduplication window ends.
The two organization creation paths send the event after their organization write completes.
A send failure does not roll back the organization. The scheduled sweep repairs it.
The durable function checkpoints each capability installation separately. A retry reloads every deterministic row and continues from stored state. The operator command calls the same reconciler and prints the same structured report.
Upgrade and rollback
ENG-2303 owns explicit upgrades and rollback. Both use the existing definition publish validation before they activate a binding.
An upgrade and a rollback are operator actions. The automatic paths never replace an active binding.
The organization event and the scheduled sweep install and repair only. They report the new status
outdated when a prior released revision serves the capability, and they write nothing for it. This is
what makes a rollback survive: the next sweep does not roll it forward.
The contract gate reads the basis: the contract the tenant moves away from. That is the active row, or, with nothing active, the newest released row that is not the manifest root. A switch cannot move away from its own target, so the root is never the basis. A gate that read the newest row would compare the target with itself whenever an earlier release had already bound the root, and a downgrade would land.
An upgrade names the capabilities that may switch. For each of them:
- Compare the target binding with the basis binding, before any write. An attach is irreversible,
because the trigger freezes
capability_bindingand refuses a delete on a bound row. A binding written on a refused switch could never be taken back. - Create or locate the inactive target revision and its tenant child definitions.
- Validate schemas, scopes, tenant references and the full dependency tree.
- Publish the target tree before activation. A failed publish leaves the old binding active.
- Switch the binding with two writes. The result is
upgraded.
The switch is two writes, not one transaction. ac-python-api reaches Postgres through PostgREST,
which cannot span two statements, and the design refuses a business-logic RPC. Each write carries the
row's expected updated_at, so a concurrent writer loses its write instead of overwriting. The partial
unique index uq_definitions_active_capability permits one active binding per organization and
capability, so the old row releases the slot before the new row claims it.
A crash between the two writes leaves the capability with no active binding. The registry then answers
inactive_executor, which is truthful, and no Run is harmed. The next reconcile pass finds the target
bound and inactive with nothing active, and it activates the target. The half switch repairs itself, and
the failure window is one HTTP round trip.
Concurrent reconcilers either observe the desired active revision or retry from the current one. They never disable the old binding before a replacement passes validation.
Deactivation runs no validation, so the release always succeeds and all the risk sits on the second write. When the target refuses activation, the switch gives the slot back, so a healthy prior revision keeps serving. Re-activation runs the full executor validation, so it can refuse too. The report then names the empty slot: a capability never goes dark in silence.
A disabled released row with nothing active refuses every install. A disable clears activation in the same database write, and the row that was serving is not recorded, so the reconciler cannot tell an operator's switch-off from a retired revision. It refuses, and the reason names the way out: enable the row, or roll back to a released revision.
A rollback runs before every manifest check, and it validates its own target. It is the way out of a disabled revision, and of an incumbent whose stored binding a later model refuses. Those are the two states that make a rollback necessary, so a manifest check must not block it. It still refuses a disabled target, so it cannot restore what an operator switched off.
A rollback names one released revision of one capability. It reads that row and runs the same guarded
switch. It creates nothing, publishes nothing and deletes nothing, so every Run and every audit row stays.
The target must be a released revision, must already hold a binding, and must be active. A disabled
previous executor is not a valid rollback target, and a disabled capability refuses a rollback outright: a
rollback must not restore what an operator switched off. The result is rolled_back.
A rollback moves direct starts only. A published parent names its child by UUID and its Run reads a frozen tree, so a parent keeps the child revision it published. Roll the parents back in the same command when the child revision is the fault.
A rollback to a revision that is not the manifest revision gates its dependants as outdated, on the
pass that writes it and on every pass after. The report still reads rolled_back for the audit, but a
dependant reads the state it will find: the current root does not exist, so it cannot publish. A rollback
to the manifest revision serves its dependants normally.
An outdated capability blocks its dependants. It writes nothing, so its current-revision root does not
exist, and a parent that names that root cannot publish. A dependant therefore reports blocked until the
child upgrades. Upgrade in dependency order, or pass --upgrade-all.
The contract version is a positive integer, independent of the executor revision.
An upgrade never lowers it. A rollback lowers it on purpose, and its report names the move.
One contract version holds exactly one pair of schemas. No code can prove that two schemas carry the
same meaning, so the switch compares them for equality rather than for compatibility. An author who
changes an input or output schema raises the contract version. Product text stays free to change.
An incompatible change needs a new contract version and an explicit migration. Old-version clients receive contract_version_conflict.
There is no side-by-side serving: one active binding per capability means one live contract version, and a
stale client fails loudly at start rather than reaching a translated contract.
A product update uses a new definition row and passes the same compatibility checks.
The generic definition editor cannot republish a bound row.
Disable makes new starts unavailable. Delete is refused while a binding, published parent or retained Run still references the definition.
Reconciliation results
| Result | Meaning |
|---|---|
current | The stored state matches the manifest. No write ran. |
installed | A write converged this capability on the manifest revision. |
upgraded | An operator upgrade replaced an active binding with the manifest revision. |
rolled_back | An operator rollback activated a released prior revision. |
outdated | A prior released revision is active. Only an operator upgrade replaces it. |
missing | Report mode found work to do. |
disabled | A released executor is disabled. An install would restore a capability an operator switched off. |
conflict | A binding outside the released revisions exists, or managed state drifted. Tenant work is preserved. |
invalid | Stored data or a contract fails validation. It is not repaired. |
blocked | A dependency is not at its released revision. |
Composition and frozen Runs
Resolve stable child IDs when publishing a product workflow. Store the resolved UUIDs in ordinary subworkflow nodes.
Store each child's capability ID and contract version with the published composition metadata.
Do not resolve a new executor from the registry at each workflow step.
An upgrade does not retarget an already published parent.
For a bound managed parent, publish a replacement parent definition and switch its binding when its child version must change.
Unbound parents retain the normal republish lifecycle.
Keep old executors runnable for those parents until the references are migrated. Do not delete their audit history.
A new root snapshot freezes the entire published dependency tree and its capability metadata.
An in-flight Run and its later child starts use that snapshot, even after a binding switch.
Children keep <parent_run_id>:<node_id> start keys and inherit the root budget and narrowed grants.
Run reads, spans and logs retain stable capability ID and contract version through API, CLI, chat and child starts.
The attribution contract defines snapshot versions, legacy Runs and metric dimensions.
Templates
| Definition | Owner | Editable |
|---|---|---|
| Platform template | platform | no |
| Organization custom definition | organization | admins |
Customization forks the platform row. A later platform change does not silently change the fork. fork copies the source's published_config into the new row's draft_config, with origin: custom, state: draft, published_config null and source_definition_id set to the source.
The copy lands in draft_config, and writing it to published_config fails the insert. definitions_published_shape is the equivalence (state = 'draft') = (published_config IS NULL), so a draft carrying a published config raises 23514. A fork is unpublished work by definition: the admin edits it and publishes it, and that publish is what fills published_config.
source_definition_id is the one reference that carries no organization, because a platform row belongs to none and a paired key would refuse it. The service is therefore the only guard: a fork source must be a platform row, or a row in the caller's own organization. A database constraint cannot express that, so the check lives in DefinitionService.fork() and it is a contract test.
A fork source must be active. fork copies published_config, and a draft holds none. draft_config is NOT NULL, so a fork of a draft would write NULL into it and fail the insert. The service refuses it first and answers not_found, because a draft of another organization must answer exactly as a missing row does and the caller's own draft needs no fork: it is already editable.
A disabled source is refused by the same method and the same answer. A disabled definition is configuration an admin withdrew, so a fork of one copies work that was switched off on purpose. The RLS policy says the same thing for a platform row: authenticated reads a platform template only while state = 'active', so a disabled one is a row no surface can list. get_fork_source therefore filters on state = 'active', and the service needs no second check.
A definition references its own organization only
A custom definition may reference only definitions of its own organization. A platform template is forked before it is referenced. So referenced_ids never holds a platform id, and the reverse lookup of a platform row is empty by construction.
This is the rule that keeps three others answerable.
| Without it | With it |
|---|---|
| the disable guard of a platform template must ask every tenant, and the reverse lookup is org scoped on purpose | the guard is one org scoped read, and a platform row has no referrers to find |
| the referrer cap counts per organization, so a platform row carries 50 referrers per tenant and no global bound | one cap, one meaning |
referenced_ids mixes tenants, so one indexed lookup answers a question the caller may not see the answer to | every id in the column belongs to the row's own organization |
It costs the one thing that reads like a loss: a platform skill fix does not reach an organization that forked it. That is the same trade the fork already makes for every other field, and the section above states it — a later platform change does not silently change the fork. The shared-fix property below is an intra-organization one: publishing a skill reaches every agent of that organization that references it, and reaches no other tenant.
A platform template is therefore reachable in exactly two ways: fork it, or run it after forking it. runs_definition_fk already pairs a run with a definition of its own organization, so the same rule was already true of execution. This extends it to references, and the two now agree.
So a fork of a platform template is a deep fork. A shallow copy of the source published_config carries its ids, so a platform workflow that names a platform agent would fork into a draft naming two rows of no organization, and this rule would refuse the publish. The admin could not repair it either: the ids name platform rows their organization does not own, so there is nothing to point them at. The alternatives are worse -- a platform template of one row forever, or a reference rule with a hole in it -- so fork copies the reachable set.
The deep fork
fork(platform workflow W)
W -> subworkflow node -> platform workflow V
-> agent node -> platform agent A -> skill S
copies V, A and S into this organization, then W,
and rewrites every id in every copy
It crosses the organization boundary, and it stops there. A reference is forked when its target belongs to no organization, and it is left alone when the target is already this organization's. Under fork-first those are the only two cases a valid graph can hold: a platform template references platform rows only, and a custom definition references its own organization's rows only. So a fork of a platform template copies the whole set, and a fork of the caller's own definition copies one row and shares the references it already had. Deep-copying the second case would mint duplicates of definitions the admin can already point at.
Every deep fork mints a fresh set. Fork the same template twice and the organization holds two independent copies of everything below it. Reusing an earlier fork by source_definition_id would put one row under two parents, so an edit made for one would change the other -- which is the coupling the fork exists to remove.
The walk reads published_config, not referenced_ids. One pair of functions reads the references out of a config and writes new ones back into it, and publish already needs the first to compute the column. Reading the column here would be a second answer to "what does this definition reference", and the two disagree the first time a seed is wrong. It is also why the seeded column stays unread: nothing walks a platform row's referenced_ids.
The writes go leaves first and the root last. There is no transaction, so the order is what bounds the damage of a crash: a root written first would name ids that do not exist yet, and a failure would leave a draft nobody can publish and nobody can diagnose. Written last, the root exists only when everything below it does.
A failure after the first insert is compensated, best effort: the drafts this fork wrote are deleted. A draft cannot run and it deletes cleanly, so the worst case left behind is rows an admin can remove. The retry mints a fresh set, exactly as a first attempt does.
A reachable definition that does not resolve refuses the whole fork, and the answer is invalid naming the id. The source template is broken -- it names a row that was removed or disabled -- and copying a subset of it would give the admin a draft they cannot publish and no reason why.
The walk carries a read budget, as the graph walk does. It is a runaway backstop and never a design limit: the workflow depth cap already bounds the shape a valid template can take.
Core code
runtime/definitions/
models.py Definition, DefinitionKind, DefinitionRef, ValidationResult
repository.py DefinitionRepository
service.py DefinitionService
validators/
base.py DefinitionValidator entry point
agent.py
skill.py
workflow.py
snapshot.py SnapshotBuilder
references.py the referrer set and the reference cap
authority.py who may write a definition
model_registry.py the provider and model pairs a publish accepts
errors.py DefinitionError, the closed outcome set
API / Builder chat
│
▼
DefinitionService
┌────┴─────────────┐
▼ ▼
DefinitionValidator DefinitionRepository
Conversational authoring enters through the same service. The front door builder chat uses one platform agent whose tools call DefinitionService. It adds no second lifecycle path.
class DefinitionRepository(Protocol):
async def get(self, definition_id: UUID, organization_id: UUID) -> Definition | None: ...
async def get_fork_source(self, definition_id: UUID,
organization_id: UUID) -> Definition | None: ...
async def list_page(self, organization_id: UUID, *, limit: int,
kind: DefinitionKind | None = None,
origin: DefinitionOrigin | None = None,
state: DefinitionState | None = None,
cursor: tuple[datetime, UUID] | None = None
) -> tuple[list[Definition], bool]: ...
async def get_many(self, definition_ids: list[UUID],
organization_id: UUID) -> list[Definition]: ...
async def create(self, definition: Definition) -> Definition: ...
async def save_draft(self, definition_id: UUID, organization_id: UUID, config: dict,
expected_updated_at: str) -> Definition | None: ...
async def references_to(self, definition_id: UUID,
organization_id: UUID) -> list[DefinitionRef]: ...
async def publish(self, definition_id: UUID, organization_id: UUID, config: dict,
referenced_ids: list[UUID],
expected_updated_at: str) -> Definition | None: ...
async def set_state(self, definition_id: UUID, organization_id: UUID,
state: DefinitionState) -> Definition: ...
async def delete_draft(self, definition_id: UUID, organization_id: UUID) -> None: ...
class DefinitionValidator(Protocol):
async def validate(self, definition: Definition, config: dict) -> ValidationResult: ...
class DefinitionService:
async def create_draft(self, kind, name, config, actor): ...
async def fork(self, source_id, actor): ...
async def update_draft(self, definition_id, patch, expected_updated_at, actor): ...
async def validate_draft(self, definition_id, actor): ...
async def publish(self, definition_id, expected_updated_at, actor): ...
async def disable(self, definition_id, actor): ...
async def enable(self, definition_id, actor): ...
async def delete_draft(self, definition_id, actor): ...
Seven of the eight are a lifecycle move, and validate_draft is the read. disable without enable is a one way door, and the surfaces page tells an admin to use disable to prevent new Runs, which reads as reversible. delete_draft is the only way an abandoned draft leaves.
create_draft is the eighth, and the surface cannot skip it. A new
definition is a lifecycle write: it sets state and origin, and both are
fields the client never writes. A router that called DefinitionRepository
directly would be the second writer of those two columns, and the authority
check would then live in two places. It takes the kind, the name and a starting
configuration, refuses a non-admin with forbidden, and answers the new draft.
It runs no validation: a new draft is empty by definition, and update_draft
already reports what is missing on the first save.
Two drafts may carry one name. No unique constraint pairs the name with the
organization, and fork already mints a second copy of a template on a second
call, by design. A name is a label an admin reads, and the id is the identity.
list_page pages on (created_at, id), and the page is a union. It reads
the caller's own rows in every state, plus the platform rows that are active,
because a fork starts from a template. RLS does not run on this path, so the
read applies state = 'active' to the platform half itself, exactly as
get_fork_source does. Ordering on name was the earlier shape; name is not
unique, so a keyset cursor on it needs an encoder of its own, and created_at
is already on the table and already paired with an id by the shared cursor. The
page reads one row past the limit, so a full page is never a truncated one.
Every repository method takes an organization_id, and it is never optional. Each call reaches Postgres on the service role, so RLS filters nothing and raises nothing. A signature that omits the tenant is a cross-tenant read on a read method and a cross-tenant write on save_draft, publish, set_state and delete_draft. Nothing else carries the tenant, so the signature does.
get_fork_source is the one read with a wider rule, and it is a separate method for that reason. A fork source is a platform row, which belongs to no organization, or a row of the caller's own organization. Expressing that as an argument to get would make the ordinary read permissive by default, and the ordinary read is the one that runs everywhere.
The list method is not called list. A method of that name shadows the
builtin for the whole class body, so every -> list[Definition] written below
it resolves to the method. mypy measures it as Function ... is not valid as a type, and RunRepository.list_page already avoids the same trap.
get_many exists because publish revalidation is a fan out. One publish loads up to 50 direct referrers and then the definitions each of them references, and a per-row read makes that hundreds of round trips inside one HTTP request. Batch the reads with one in.() filter per level, exactly as RunRepository does, and keep them inside the FILTER_BATCH_SIZE the run repository already measured.
expected_updated_at gives optimistic concurrency, on a draft save and on a publish, and on neither of the three state moves. A stale writer reloads. It does not overwrite the work of another admin. The repository returns None for a stale write rather than raising, because the caller always reloads. No distributed lock and no revision subsystem is added.
disable, enable and delete_draft carry no token, and that is deliberate. disable is the emergency switch. An admin who presses it during an incident must not be answered stale because a colleague saved a draft a second earlier, and the token buys nothing there: the write sets one column and overwrites no authored work. The race it leaves is bounded and already covered. A publish that adds a referrer between the disable guard's read and the disable's write leaves an active referrer pointing at a disabled definition, and enable revalidates, which is the same place every other deferred referrer break surfaces.
A BEFORE UPDATE trigger moves updated_at on every write, so a publish and a disable each invalidate an editor's token. That is correct: both change what a later save would build on.
The write is one conditional statement, because there is no transaction. ac-python-api reaches Postgres through PostgREST, so a save is PATCH ...?id=eq.<id>&updated_at=eq.<ts> and an empty result body means stale. This is the same shape RunManager uses for a lifecycle transition. See runtime execution.
A draft may be incomplete. Full validation runs at publish.
The draft patch replaces a field, and never merges into it
update_draft takes a shallow patch over the top level fields of the kind. A named field is replaced whole. An absent field is untouched. An explicit null clears the field.
patch {tool_ids: ['crm.read_company']} -> tool_ids becomes that one element
patch {} -> the surface refuses it with 400
patch {context_policy: null} -> the field is cleared
An empty patch is refused, and it is not a no-op. definitions_updated_at
is a BEFORE UPDATE trigger, so it moves updated_at on a write that changes
no value. A save of {} would therefore answer ok, change nothing, and make
every other admin's token stale. The surface refuses the empty object, so the
one write this service makes always carries a change.
A deep merge cannot remove anything, and removal is the first edit an admin asks for: drop a tool, drop a skill, shorten a node list. A recursive merge would make tool_ids grow on every patch and never shrink, and the builder agent would have no way to say what it means. A workflow node list is one field under this rule, so an edit to one node sends the list.
update_draft runs the same candidate validation publish runs and returns its ValidationResult, advisory: an invalid draft still saves. It is the candidate half alone. A draft is never in a referrer set, so the referrer revalidation and the referrer cap have nothing to read, and a save that ran them would fan out to fifty definitions on each keystroke of a builder agent. validate_draft is that one validation with no write, for a surface that checks before it commits. One code path answers both, so a draft can never be told it is valid by one method and invalid by the other. A draft is expected to be incomplete, and the result is what tells the builder agent what is still missing.
The closed outcome set
Like StartRunResult, the service answers with a value and never an exception, because four surfaces branch on it.
ok the write landed
stale expected_updated_at did not match; reload and retry
invalid validation failed; the result names every error
definition_in_use disable refused; an active definition references this one
referrer_limit publish refused; it would exceed the direct referrer cap
not_a_draft delete refused; a published definition is disabled, never deleted
not_published disable or enable refused; the definition has never published
forbidden the actor may not publish in this organization
not_found no such definition for this organization
not_published is the ninth member, and it closes the state moves the other eight could not answer. disable and enable on a draft are reachable from any surface that lists definitions of every state, and none of the other members says what happened: not_a_draft is the inverse, not_found is a lie the Builder disproves on its next tab, and invalid names errors that do not exist. It maps to 409, exactly as not_a_draft does.
Three more calls are refused by members already here. fork of a draft answers not_found, per the templates section above. publish of an empty draft_config answers invalid, because schema validation is the first check and an empty config fails it. disable of a definition already disabled, and enable of one already active, answer ok and write nothing: both are idempotent, because the surface behind them is a toggle and a second press must not raise.
Publish rules for references
A definition can reference other definitions. An agent references skills. A workflow references agents, tools and subworkflows.
Two rules keep the graph valid.
- A referenced active definition cannot be disabled. Return
definition_in_use. Never cascade the disable, and never defer the failure to Run time. - A publish revalidates every definition that references it. Publishing agent A can remove a tool that workflow W depends on. Publishing subworkflow S can push workflow W past the depth cap. Validating only the definition being published leaves the break to be found at Run time.
If a referring definition would break, the publish fails and names it. The admin then fixes both, or forks.
Only a regression refuses the publish. Validate each referrer twice, against the current published_config and against the candidate, and refuse only a referrer that passes the first and fails the second. A referrer that already fails against the current config was broken before this publish, and this publish is not what broke it: a deploy that retires a tool from the registry, or a validator that gets stricter, breaks referrers in place with no write at all. Refuse on the raw result instead and one such deploy makes every shared definition in the organization unpublishable at once, and each refusal names a definition whose fix the publisher may not own.
This is the same rule as the two below it. Put the refusal on the write that breaks the bound, not on the write that stands next to a bound already broken. A referrer that is already invalid surfaces where every other deferred break surfaces: its own next publish, and enable.
Bound the fan out. One shared agent may be referenced by many workflows, and validating all of them inside one HTTP request is how a publish becomes a timeout. Publish revalidates direct referrers only. Validation is deterministic and cheap, so the bound is a latency ceiling rather than a correctness one.
An indirect referrer is not covered, and the run time check is what covers it. The claim that a break propagates one level per publish holds only when the middle definition republishes, and nothing forces it to. Take V -> W -> S with the depth cap at three. Publishing S at depth two makes depth(W) three, which is legal, so W's revalidation passes and the publish lands. depth(V) is now four and V is broken, and no write anywhere is scheduled to discover it.
So the bound is honest about what it buys: direct revalidation catches the break the publisher can act on, and runtime execution re-checks the depth when a subworkflow node starts a child Run, which is where V fails. That re-check is not an optimisation and it is not belt-and-braces. It is the only mechanism that sees this case.
The cap is enforced where the count grows
A publish may not make its definition the 51st direct referrer of anything it references. The refusal is referrer_limit, and it names the definition that is already at the cap.
The obvious placement is the opposite one, and it is a trap. The count grows when a referrer publishes, not when the referenced definition does. Enforce the cap on the referenced definition and an organization reaches 51 referrers without that definition ever being published, and it can then never be published again. Its own fix is refused by the same rule. Nothing in the product releases it.
Checking it on the growing side moves the refusal to the exact write that would break the bound, and the shared definition stays publishable at every count.
An organization that reaches 50 has a shared definition that deserves a fork, and the refusal says so.
Count the referrers of the target excluding the definition being published, and check only the targets this publish adds. A re-publish grows nothing: agent A already references skill S, so A is already in S's referrer set, and a naive count >= 50 refuses A's next publish while S sits at exactly 50 with A among them. That is this section's own trap in a third shape — a definition that can never publish again, and nothing in the product releases it. So the check is over candidate_referenced_ids - current_referenced_ids, and for each target it asks whether the referrers other than this one already number 50.
The referrer set is published_config only
A referrer is a definition whose published_config names this one. A draft is not runnable, so it is neither revalidated nor counted.
One set, three rules, three different filters. They are easy to blur into one query, and each one answers a different question.
| Rule | Which referrers | Why |
|---|---|---|
| the disable guard | active only | a switched-off referrer is not in service, so nothing breaks |
| the referrer cap | active and disabled | enable returns a disabled referrer to service, so it still holds a slot |
| publish revalidation | active only | see below |
So the repository has one lookup and not two, and DefinitionRef carries state. A signature that answers a bare list or a bare count can express none of the three rows above, and a caller that wants the cap would have to ask for the wrong set and hope. references_to answers every referrer with its state, the service applies the filter its rule names, and the cap is a len() over the two states it counts. Three filters in one place read as three rules. Three filters spread over two repository methods read as a bug.
A disabled referrer must not refuse a publish, and enable is the check that covers it. The lifecycle rule above already says enable revalidates, so a disabled referrer cannot silently return to service broken. Refusing the publish as well is the trap the referrer cap section names, in a second shape: disable workflow W, and agent A can never again publish a change W would reject. A is live, W is switched off, and nothing in the product releases A. delete_draft cannot help, because W is published and a published row never deletes.
So the refusal moves to the write that would put the broken definition back in service, exactly as the cap moves to the write that grows the count.
Publish validates before it writes
There is no transaction. ac-python-api reaches Postgres through PostgREST, which cannot span two statements, so a publish that wrote first and validated second would have nothing to roll back. A failing referrer would leave an invalid graph behind and report an error at the same time.
So the order is fixed.
publish(definition, expected_updated_at)
-> validate the CANDIDATE config
-> load the direct referrers, and validate each one AGAINST the candidate
-> any failure -> refuse, name the referrer, write nothing
-> PATCH ...?id=eq.<id>&updated_at=eq.<expected>
published_config, published_at, referenced_ids, state
empty result -> 'stale'; reload and retry
The refusals before that walk are ordered, and the order is behaviour a test reads.
1 not_found the read is org scoped, so a platform row and another tenant's row land here
2 forbidden the actor is not an admin of this organization
3 not_published disable or enable on a draft
4 invalid the candidate, then a referrer that regresses against it
5 referrer_limit the cap, over the targets this publish adds
not_found is first because every later answer says that a row exists. forbidden precedes each state answer for the same reason in the other direction: an actor who may not publish learns nothing about the state of a definition. The two cheap checks therefore run before the pass that loads fifty referrers.
The cap runs before the referrer revalidation, because it is a len() over one indexed read per added target and the revalidation is the expensive pass. A publish refused by the cap costs one round trip per target and no validation at all.
One publish validates up to a hundred times, so it holds one loader for all of them. Fifty direct referrers, each validated against the current config and against the candidate, is a hundred validation passes inside one HTTP request, and a workflow pass walks the reference graph. Build one loader at the top of publish, give it one cache keyed by definition id, and pass it to every pass. Without the cache the same subworkflow is read once per pass, and the publish becomes the timeout the fan-out bound exists to prevent.
published_at is in that list, and leaving it out fails the first publish of every draft. definitions_published_at_shape is the equivalence (published_config IS NULL) = (published_at IS NULL), so a row that gains a config without a timestamp raises 23514. The column is not a display field; it is half of a constraint.
Every publish writes it, so it reads as "last published" and never as "first published". created_at already holds the other date. A publish that wrote the timestamp once would leave an admin looking at a month-old date beside configuration changed this morning, and the skill freeze rule below is the case where that date is exactly what an admin checks.
expected_updated_at is an opaque token, and the layer that must not parse it is the HTTP surface. The filter travels in the URL as text, and PostgREST parses it back to timestamptz before it compares. So an equivalent spelling matches: a different UTC offset, padded trailing zeros, a space in place of the T, and a Python datetime round trip that re-pads .18 to .180000 all match the row. Measured against the local stack, on one row whose updated_at is 2026-08-21T20:31:36.180302+00:00:
2026-08-21T20:31:36.180302+00:00 the value as returned matches
2026-08-21T22:31:36.180302+02:00 another offset matches
2026-08-21 20:31:36.180302+00 a space, a short offset matches
2026-08-21T20:31:36.180302000+00:00 padded matches
2026-08-21T20:31:36.1803+00:00 four digits MATCHES NOTHING
2026-08-21T20:31:36.180+00:00 three digits MATCHES NOTHING
Only a loss of precision breaks it, and one client type loses precision by construction. timestamptz carries microseconds and a JavaScript Date carries milliseconds, so new Date(token).toISOString() truncates to three digits and the filter then matches no row: every write answers stale while the data is fine. PostgREST also strips trailing zeros on the way out, so .180000 is returned as .18 and the truncation is invisible in the value the client holds.
The Agent Builder is a browser client and PATCH /api/v1/agentic/definitions/{id}/draft is its route, so this is the path the defect sits on, not the Python one. The rule therefore belongs on the surface: the API returns expected_updated_at as a string, documents it as opaque, and no client parses it into a date type. The repository types it str for the same reason. A Python service that does parse it stays correct, which is exactly why the rule cannot be tested by watching Python work.
The token carries a +, and a query string reads + as a space. The filter goes in the URL as updated_at=eq.2026-08-21T20:31:36.180302+00:00, so the client must send %2B or PostgREST parses 2026-08-21T20:31:36.180302 00:00 and the write answers stale every time. No repository in ac-python-api filters on a timestamp today, so there is no call site that proves the client library encodes it. Prove it with an integration test against the local stack, on a token read straight back from the row. A unit test over a mocked client proves the mock.
One race survives, and it is bounded. A referrer may publish between the check and the write, so it is validated against the old config and never against the new one. Two backstops catch what that leaves, and both already exist: the depth cap is re-checked at run time, and a reference to a step that did not run fails the node with unresolved_reference. A third publish of either definition repairs the record. V1 does not add a lock for this, because the window is milliseconds and the run time answer is already correct.
The reference set is a column, not a scan
A reference is to another definition: the skill_ids of an agent, and the target of every agent and subworkflow node of a workflow. Reading them back the other way is one query that four rules depend on — the disable guard, the referrer revalidation, the referrer cap, and cycle detection.
tool_ids are not in it. A tool is a registry entry keyed by name, not a row of agent.definitions, and V1 has one scope vocabulary which is that name. Tool existence is a registry lookup in the validator, and a tool id can never enter a UUID[]. See tools and integrations.
A JSONB scan cannot serve the reverse lookup. agent.definitions has no GIN index, and a reverse lookup over a JSONB document has no index to use at any table size.
So publish computes the set and stores it.
agent.definitions
referenced_ids UUID[] the definition ids published_config names, computed at publish
The column is not called references. REFERENCES is a reserved word in PostgreSQL, so CREATE TABLE ... (references UUID[]) is a syntax error and only "references" parses. It would be the one quoted identifier in the schema, and every later index, grant, migration and hand query would have to keep quoting it. ac-backend writes unquoted lower case identifiers, so the column takes a name that needs no quotes.
GIN on referenced_ids makes references_to one indexed query. The column is derived, so it is written only by publish, and never by hand. draft_config contributes nothing to it, which is what keeps a draft out of the referrer set by construction rather than by a filter someone must remember.
A seeded platform template is the one row publish never writes, so its referenced_ids is seeded too. The seed writes the column, and a contract test proves each seeded row's column matches its own published_config. Nothing reads it today. The deep fork walks the config rather than the column, and a platform row has no referrers to find, because a custom definition references its own organization only. The rule stands so that the column never disagrees with the row it belongs to, which is what a later reader would trust.
The lookup takes an organization_id, and it is not optional. Every read and write on this schema uses the service role, so RLS filters nothing and raises nothing. Without the argument, references_to(id) reads across every tenant: the cap becomes global, so one organization's fiftieth workflow refuses another's publish, and the refusal names a definition in an organization the caller cannot see. The signature carries the tenant because nothing else will. The fork-first rule above is what makes the answer complete as well as scoped — every id in the column belongs to the row's own organization, so an org scoped lookup can miss nothing.
A Postgres array takes no foreign key, so nothing stops referenced_ids dangling by itself. Two existing rules do it instead: validation refuses a reference to anything but an active definition, and a published definition is disabled rather than deleted. A draft is the only row that leaves, and a draft can never be referenced. Do not add a trigger for this; add a contract test.
This column is additive and it is not in migration 20260819120100. It belongs to the ticket that builds this service, and that migration must decide the read grant explicitly. agent.definitions grants authenticated an enumerated column list, so a new column is unreadable until a migration names it. referenced_ids stays out of the list: it is derived, the service is its only reader, and the reverse graph of one organization's definitions is not something a member needs.
Agent definition
model
instructions
output_schema? JSON Schema Draft 2020-12 for one object
context_policy
tool_ids[]
skill_ids[]
budget_defaults max_segments, max_agent_turns, max_tool_calls, max_run_duration_s, max_cost_cents
The agent receives only its explicit tool set. A skill never widens it. The agent validator holds that rule, because a skill has no agent at publish time. See the skill definition.
model resolves in the model registry at publish. A definition may otherwise publish any string, and the failure arrives at the first Run of a definition an admin believes is valid. The check is one lookup and it is deterministic, which is the standard every other check on this page meets.
The registry is a constant in the definitions package, and it is not a table. V1 supports a fixed set of provider and model pairs, a deploy adds a pair, and no admin edits one. A table would need a lifecycle, a migration and a surface for a list that changes when the code changes. ModelConfig already carries provider and model, so the registry is the set of legal pairs plus the per-pair defaults the snapshot freezes.
output_schema is optional. When it is present, it must be a valid JSON Schema Draft 2020-12 document with type: object. Publish freezes the schema in the Run snapshot. The runtime requests JSON, validates the returned object locally against the frozen schema, and writes it to RunResult.output. An agent workflow node then exposes that object as steps.<id>.output.
The runtime does not send this plain schema as a provider-native response format. OpenAI and Anthropic use different response envelopes. JSON mode plus local validation keeps one portable definition contract.
When output_schema is absent, the agent keeps the text result contract. Its text becomes RunResult.summary, and RunResult.output stays empty. This keeps existing agent definitions unchanged.
budget_defaults becomes the Run ceilings. Policy limits cap them. See runtime execution.
max_run_duration_s is capped at 30 days by validation. Idempotency retains a tool claim for 30 days because the claim is the replay journal and it must outlive the longest Run any definition may set. That retention is a per scope constant, so it can only take the ceiling if a ceiling exists. Without this cap a definition could declare a 60 day Run whose journal is forgotten halfway through, and a later segment would repeat an effect it already made.
Skill definition
description
instructions
tool_ids[]
V1 stores short procedure text in Postgres. skill.tool_ids must be a subset of agent.tool_ids, and the agent validator checks it. A skill publishes on its own, with no agent in hand, so the check cannot live in the skill validator. The referrer rule closes the other direction: publishing a skill revalidates every agent that references it, and an agent that no longer covers the skill's tools fails there.
A skill renders at the freeze, so a skill publish reaches an agent that did not republish. SnapshotBuilder reads the skill's current published_config, so a new Run of agent A uses the skill published a minute ago, whatever A's own published_at says. This is intended: a skill is shared procedure text, and forcing every referring agent to republish would make a wording fix an N-definition change.
State it to an admin, because it is the one place the publish boundary bends. An in flight Run is unaffected, since its snapshot already holds the rendered text.
Use a workflow when the order is fixed. Skill instructions are flexible guidance. They are not executable nodes, and the Run foreign key carries kind, so a skill cannot be the definition of a Run.
Workflow definition
A small declarative tree. The node types are listed in agentic runtime.
Each node has a stable ID, a typed input or reference, and an optional continue_on_error flag.
Two fields that were declared here are gone, and both were measured.
| Field | Why it is absent |
|---|---|
retry | The Inngest Python SDK sets retries per function, and step.run takes no count. A node cannot carry one. A transient fault already raises out of the node body and reaches the function retry, which replays every memoized step for free, so the field named a second retry policy the platform cannot honour. |
result_key | steps.<id>.output is keyed on the node ID, so a result key is a second name for one value. |
A workflow may declare one result projection. The optional top-level
result field is an object of literals and the same marked references that a
node input uses. The runtime resolves it only after the complete workflow
succeeds. For example:
{"result": {"smart_feed": {"$ref": "steps.compile-smart-feed-observations.output"}}}
This is not a second name on a node. It is the small public result of the whole
workflow. A workflow that omits it keeps the existing result map with one key
per output-producing node. A partial success also keeps that diagnostic map,
because the selected final node might not have run. A selected reference that
cannot resolve after a complete walk fails the Run with
unresolved_reference; it never becomes null.
Publish validation requires result to be an object. It applies the same
reference depth, namespace, known-node and output-producing-node checks as a
node input. The snapshot freezes the object with the root and ceilings.
A reference is marked, because a bare string cannot be. A declared input holds literals, and "input.campaign" is a legal literal. So a reference is a one key mapping and nothing else in the language is:
{"query": {"$ref": "steps.classify.output.company_name"}, "limit": 25}
The rejected alternative was a template over the string, which is an expression language by another name. The page refuses one for a condition, and the same reason holds here.
A node ID is unique across the whole workflow, not inside its container. steps.<id> carries no path, so a per container ID cannot resolve. A subworkflow keeps its own ID space: a parent sees one value at the subworkflow node, and never a node inside it.
A node ID is at most 410 characters, because it becomes a claim key. A node ID is a step path, and two keys are built from one: <run_id>:<node_id>:<args_hash>, which the tool journal writes and which an approval node writes. Both columns hold 512, and a UUID and a SHA-256 digest take 102 of them. Unbounded, a workflow publishes and then fails mid Run, because the key is built at the node and not at the publish.
Three node types produce a steps.<id>.output, and three do not.
| Node | steps.<id>.output |
|---|---|
tool | the bounded ToolResult.output |
agent, subworkflow | the child Run's RunResult.output |
sequence, parallel, branch | nothing. A container produces no value of its own |
A container coordinates; it does not compute. Name the child that produced the value, not the container around it. Validation refuses steps.<container_id>.output, because the alternative is a run time null in a workflow that looked valid.
A retry cannot repeat an effect that already happened. The idempotency journal keys on the node ID and the argument hash, so a replayed tool node with the same arguments reads the stored result rather than calling again. That is the correct behaviour, and it is why the retry the function already gives is enough.
| The node failed on | The function retry |
|---|---|
| a platform fault, or a retryable upstream error | helps. The claim was released, so the call really runs again |
a business failure such as not_found | cannot help. The claim completed, so every attempt reads the same answer |
A business failure needs continue_on_error or a branch, and never a retry. That is why the node body returns it as a value rather than raising: a raise reaches the function retry, which cannot change the answer and which ends every sibling branch of a parallel node.
A reference to a step that did not run fails the node. A branch picks one child, and continue_on_error lets a node settle with no output, so steps.<id>.output.* can name a step that never produced anything. Validation checks that the name exists; it does not prove the step is reachable on every path, and V1 does not add a reachability analysis to find out.
At run time the node fails with unresolved_reference, naming the step. It fails rather than substituting a null, because a workflow that silently sends an email with an empty body is worse than one that stops.
A fan out step declares two numbers. A tool or agent step carries max_fanout, the total items, and fanout_concurrency, how many items can be in flight.
Phase 4 runs wide tool steps only. A wide tool must have workflow_allowed = True and be read-only (side_effects = "read"). Write and send fan outs wait until partial effects and approvals have one explicit recovery rule. An agent step keeps both fields at one until the runtime can start and join many child Runs. Publish refuses every wider node that the executor cannot run.
A workflow that must write many rows uses a batch write tool, not a wide step. One width-one call carries the whole bounded list. The handler writes each row on its own idempotent key and opens no transaction over the batch, so a retry of the same call resolves the same rows and completes the missing ones. That is the recovery rule a wide write still lacks.
A wide tool input is one reference to a list of complete argument mappings. It does not add an item template or an item namespace:
{
"type": "tool",
"id": "find_people",
"tool": "research.search_people",
"input": {"$ref": "steps.rank.output.people_search_inputs"},
"max_fanout": 200,
"fanout_concurrency": 20
}
Each list item is the full public argument mapping for one ToolInvoker call. An empty list succeeds with an empty list. A non-mapping item fails the node before any call starts. A completed node returns one ordered envelope per input item. A ceiling returns the dense prefix through its first stopping input position, as runtime execution defines:
[
{"ok": true, "data": {"company": "Example"}},
{"ok": false, "error": {"code": "not_found", "message": "No company matched"}}
]
A business failure settles one item and does not fail the node. continue_on_error does not change item handling. It applies only if the whole node fails. A tool step with max_fanout = 1 keeps the existing mapping input and the existing single result.
This shape keeps the language small. The step before the fan out builds the argument list. The fan out does not need a template, an alias or an item.* reference namespace.
ceil(max_fanout / fanout_concurrency) x timeout_s
<= the step budget - FANOUT_STEP_HEADROOM_S
timeout_s bounds one complete fan-out item. The wide scheduler applies it around the item worker, including ToolNodeCaller.call and its result envelope. Argument validation, policy, metering, the handler and result handling all fit inside the number used by the formula. The inner handler timeout stays as a second guard. Without the outer guard, policy or persistence time would sit outside the proof.
FANOUT_STEP_HEADROOM_S is 10 seconds. It covers list checks, task scheduling, batch transitions and final serialization. The executor also applies the full step budget around the node. The fixed reserve makes equality in the item formula safe and the outer guard protects a stale snapshot.
"The step budget" is the worker's step ceiling, and no node declares one. Runtime execution owns the number and the rule that sets it. Do not restate the number here. This check puts a fan out inside the same ceiling. It is not max_run_duration, which bounds the whole Run and not one step.
The retry count is not in the formula. A retry replays the function and the fan out step. Each attempt is a separate step execution. The step budget bounds one execution. Multiplying by retries would reject a valid definition.
Retries increase the Run's latency. max_run_duration includes that elapsed time.
Registry validation already checks that one call fits the step budget. That is not enough for a step making hundreds of calls: a wide fan out passes every per-call check and then blows the budget as a whole.
Two numbers rather than one, because each answers a different question and a single number gets one of them wrong.
| Number | Bounds | Read by |
|---|---|---|
max_fanout | how much work the step may take on | the step budget check above |
fanout_concurrency | how hard the step hits a vendor at one instant | the vendor arithmetic in tools and integrations |
A width of 200 is ten batches at the maximum concurrency of 20. Multiplying by the width alone would reject a valid step, and treating all 200 as one batch would overload one worker.
Both values are strict positive JSON integers; a boolean, string or decimal is invalid. Both default to one. MAX_FANOUT is 200. MAX_FANOUT_CONCURRENCY is 20. Concurrency cannot exceed the declared width. The defaults keep a step with no fan out declaration unchanged and do not turn one wide declaration into an unbounded burst.
The declared output must fit one memoized step. Inngest accepts at most 4 MiB from one step. Validation gives tool data 3 MiB and keeps 1 MiB for the item envelopes and node result. It uses the tool's max_output_bytes, or the 32 KiB platform tool limit when the tool declares none:
max_fanout x effective max_output_bytes <= 3 MiB
The executor also measures the final serialized envelope before it returns. The
envelope must fit Inngest's 4 MiB step-output limit. A stale snapshot or an
unexpectedly large error cannot turn into an Inngest 413. The executor fails the
node with fanout_output_too_large and no truncated success.
The item-time and 3 MiB node-data checks apply only when max_fanout > 1. A width-one tool keeps the registry's existing timeout rule and its existing single-result shape.
The workflow must fit Inngest's 32 MiB function state too. Validation gives declared node data 24 MiB and keeps 8 MiB for the event, item and node envelopes, and Inngest metadata. It sums every leaf conservatively, including both sides of a branch:
one tool item effective max_output_bytes
one wide tool max_fanout x effective max_output_bytes
agent/subworkflow the 32 KiB RunResult output limit
sum of every leaf <= 24 MiB
This is a publish check, not a run-time truncation. A workflow over the total returns workflow_state_too_large and does not publish.
ENG-2201 does not open the publish gate. It adds the node fields and every static check above, then returns fanout_not_available for an otherwise valid wide node while WIDE_TOOL_EXECUTION_READY is false. ENG-2202 lands the scheduler and changes that constant to true in the same release. A declaration cannot reach a width-one executor between the two tickets.
A wide tool takes no item claim. Phase 4 permits reads only, and a read never enters the tool journal. Every call keeps the node ID as its step path, and no item key is added. MAX_NODE_ID therefore stays 410. Equal argument mappings are equal read calls and can both run.
max_fanout is a ceiling the step enforces, and a step over it fails. A run time list longer than max_fanout fails the node with fanout_exceeded, naming the declared number and the number of items. It does not silently process the first max_fanout of them. A truncation looks like a success, and a workflow that researches 200 of 500 companies and reports success is the failure this number exists to prevent. An author who expects a longer list raises the declaration, and the budget check tells them whether it fits.
Inputs may reference only allowed namespaces:
input.*
steps.<id>.output.*
V1 has no loops, no expression language, no dynamic fan out node, no private schedule and no private event trigger. Schedules and events are trigger rows.
Depth counts workflows, and the cap is three.
depth(a workflow with no subworkflow node) = 1
depth(a workflow) = 1 + max(depth of each referenced subworkflow)
cap = 3
Cycle detection runs before this function, and the order is not a preference. The recursion has no cycle guard, so a workflow that reaches itself recurses until the stack ends. A publish that answers RecursionError instead of invalid gives the admin nothing to fix. Detect the cycle, name it, and only then measure depth.
A cycle is possible between workflows and nowhere else, so the walk is over subworkflow targets only. An agent references skills, a skill references nothing, an agent node targets an agent and a subworkflow node targets a workflow. No path leaves a workflow and returns to one except through a subworkflow node. Write the walk over that one edge rather than a general traversal of referenced_ids: the general one visits agents and skills to prove something their shape already proves, and it is the version that later grows a case nobody needs.
So a top level workflow may reach two levels of subworkflow below it, and no more. Container nesting is not counted here: a sequence inside a parallel inside a sequence costs nothing, because it starts no child Run. Containers carry their own separate cap of 10, which exists only to refuse a document no person authored.
The same function is called twice, and it must be one function. Publish checks it, and runtime execution checks it again when a subworkflow node starts a child Run, because a subworkflow published after the parent was validated can deepen a tree that is already running. Two implementations of one cap disagree once, and the disagreement is a workflow that publishes and then fails mid Run.
The branch node
branch shipped with Phase 1, and wait and approval with Phase 2. Their shapes are fixed here together, so the workflow model reads in one place.
branch node
id
cases[] condition -> child node
default a child node, required; see the fourth rule below
# the email sequence chooses by what the reply said
cases
- when steps.classify.output.intent == 'interested' -> book_meeting
- when steps.classify.output.intent == 'unsubscribe' -> suppress
default -> follow_up
A condition is a ConditionEvaluator expression, and it is the same evaluator Policy and Triggers use. There is no Python, SQL, CEL or Rego, and no model evaluated expression. A third caller of one evaluator is the whole reason it is shared, and it is why the evaluator sits at src/agentic/shared/conditions.py rather than inside any one plane. See policy and governance.
A condition is stored as data, not as text. The line above is how the builder renders it to a person. What the definition holds is a small tree, and the evaluator walks it.
{"op": "and", "of": [
{"op": "eq", "path": "steps.classify.output.intent", "value": "interested"},
{"op": "gt", "path": "steps.score.output.value", "value": 70}
]}
Text would need a lexer and a parser, and a parser is the expression language this page already refuses. It also grows: the first request the grammar cannot serve arrives as a function, and the second as a cast. Data has no such edge, and a JSON schema validates it at publish with no code of its own.
The operator set is closed, and it is the one policy and governance names.
eq ne equality
lt lte gt gte numeric comparison, on numbers only
in not_in membership in a declared list
exists the path resolves to a value that is not null
and or not composition
path reads one fact through a dotted lookup. The three planes differ only in the map behind it: a workflow reads input.* and steps.<id>.output.*, policy reads its principal, argument and target facts, and a trigger reads the event.
The fact map a branch reads is the workflow's own namespaces, and nothing else.
input.*
steps.<id>.output.*
It reads no CRM row, no policy rule and no clock. A branch that needs a business fact puts a tool node in front of it, so the fact is a step with a span behind it rather than a hidden read.
Four validation rules keep a branch honest.
- Exactly one case is taken. Cases are evaluated in order and the first match wins, so an unreachable case is a configuration error and not a run time surprise.
- Every condition resolves inside the allowed namespaces. This is the same check every other reference gets.
defaultis required when no case is provably total. V1 proves nothing, sodefaultis required, always. A branch with no matching case and no default is a Run that stops with no result, and the author intended one of the children.- A child is a node of this workflow. A branch selects; it never starts something the workflow does not declare.
The taken child runs. The others do not, and their steps.<id>.output never exists. That is exactly the case the reference rule below covers.
The wait node
waitandapprovallanded with Phase 2. They need the approval plane and the Inngest wait helpers. Their shapes are fixed here so the workflow model is readable in one place, and the validator covers all eight node types.
A wait node holds for an event or for a delay. It is the only node an author configures with something the platform must correlate at run time, so its shape is fixed here.
wait node
id
mode event | delay
event_type the platform event name, when mode is event
match correlation keys, from input.* or steps.<id>.output.*
timeout_s required whole seconds, at least one
on_timeout continue | fail default continue, mode = event only
# the email sequence waits for a reply on the thread it just sent
mode event
event_type email.reply_received
match thread_id: steps.send.output.thread_id
timeout_s 259200 # 3 days
on_timeout continue # draft the follow up
The timeout is whole seconds, and the key names the unit. budget_defaults already writes max_run_duration_s, so an author reads one spelling in both places, and the config carries no duration a reader must parse. The SDK also refuses a wait under one second and one that is not a whole number of seconds, so a fractional value is refused at publish rather than in a raise the Run cannot survive.
match becomes the Inngest wait expression over async.data.*, and triggers publishes every platform event under platform/<event_type>. That page also owns the no-wait-index rule and the event type version rule.
What the expression holds, and what the router owes
async.data.organization_id == "<the Run's organization>"
&& async.data.data.<match key> == <the resolved value, as JSON>
⚠️ The tenant clause is required, and the match is not one. An author can correlate on a low cardinality field — a stage, a status, a campaign name — and an expression holding only those matches another organization's event and wakes this Run. The approval wait carries the same clause for the same reason.
The two levels are the envelope. EventRouter sends the whole PlatformEvent as the Inngest event data, so organization_id sits at the top of async.data and the business fields sit one level below it, under async.data.data. A router that sends the business fields alone matches nothing, and every wait then times out reporting that nobody sent an event somebody did send.
Every value is placed as JSON, and never concatenated. A correlation value is run time data, so a value holding a quote would otherwise close the literal and add a clause of its own.
⚠️ A correlation is a scalar. A match reference can resolve to a list or a mapping, and CEL equality over one compares the whole structure: a field the producer adds later makes the event stop matching, silently, on a wait that already ran for months. Validation cannot see it, because the value exists only at run time, so the node fails with wait_match_not_scalar. A value that resolves to null fails earlier still, with unresolved_reference — a correlation on null matches every event holding a null field.
Six validation rules keep a wait from leaking a Run:
mode = eventneeds anevent_type, at least onematchkey, and a timeout.- Every
matchvalue must resolve inside the allowed namespaces. A correlation the workflow cannot produce never matches. - A timeout is required, always. A wait with no ceiling is a Run that never ends, and
max_run_durationshould be the backstop, not the mechanism. on_timeoutis refused on adelaynode. A delay ends on its timeout every time, sofailthere reads "always fail" andcontinuestates the default twice.continue_on_erroris refused on awaitnode.on_timeoutalready answers what happens when the wait does not resolve, and two knobs over one question disagree the first time an author sets both.- At most one child subtree of a
parallelnode parks. Awaitmay sit in one child subtree, and no other subtree of that container may hold awaitor anapproval. The reason is below, and it is the one rule of the six that is about the Run row rather than the node.
Why two waits under one parallel are refused
resume() keeps a Run asleep by re-reading the pending rows in agent.approvals. That is what lets an approval inside a parallel node work: two branch approvals are two rows, the first answer re-aims waiting_ref_id at the second, and the Run stays waiting until both are answered.
An event wait files no row of its own. That is the design triggers states, and a provider job row does not change it: resume() counts pending approvals and counts no provider job. So resume() counts zero outstanding waits and wakes the Run on the first event, while the second branch is still parked.
⚠️ That is not a wrong label. It ends the Run. The reaper reads queued and running and nothing else, and it fails a running Run that is quiet past the abandon window with worker_lost. A parked branch writes no heartbeat, so a Run woken early is reaped inside the abandon window, and the wait a person is owed never resolves.
The rule is therefore structural and checked at publish: walk each parallel node, and refuse it when a wait appears in more than one of its child subtrees. Two waits in one subtree are sequential, and they are allowed.
⚠️ It reaches an approval in a sibling subtree too. A container holding one wait and one approval has one wait in one subtree and passes the count. The person answers the approval, resume() counts zero rows, and the parked branch is reaped exactly as above. So the refusal is over both node types: a parallel whose subtrees hold a wait holds no other parking node beside it. See a wait and an approval never share one parallel.
It costs the author nothing, because the shape that needs many concurrent waits already has one. A batch of people is a parallel of subworkflow nodes, one child Run each, and a child Run owns its own row and its own waiting status. The email sequence is written that way for other reasons already.
The restriction is on the Run row and not on the node, so it lifts the day a wait is countable. A container of approval nodes alone is unrestricted for exactly that reason: every one of them is a row resume() can count.
The timeout is clamped to the Run deadline
ApprovalService.create() caps expires_at at the Run's own deadline, and a wait node takes the same cap: the timeout handed to Inngest is min(timeout_s, run_deadline() - now()), floored at one second.
⚠️ Without the clamp nothing ends the Run. A three day wait on a Run whose max_run_duration_s is one day parks past the ceiling. The reaper never reads a waiting row, the wall clock gate runs before a node and not during one, and the Inngest timeout is the only writer left — three days after the Run should have stopped. A Run whose remainder is already spent settles the node at once. An event wait settles through on_timeout, and a delay wait ends and the walk goes on, because a delay declares no on_timeout.
A wait catches only what arrives after it registers
The segment approval wait reads the approval row once before it registers the wait, and once again on the timeout. Those two reads close the gap between the row being written and the waiter existing, because a person can answer inside it.
Most event waits have no row to read. The correlation comes from a step that already committed — steps.send.output.thread_id is written before the wait node starts — so a reply that lands in that gap reaches no waiter and Inngest drops it.
A wait whose correlation names a durable row does read it, twice. The check step runs before the wait registers and the timeout step runs after it ends, and both read the row the correlation names. This is the pair the approval wait already uses, over a second row type.
step <id>.mark plan the hold, mark the run `waiting`, open the span
step <id>.check read the row; a terminal state skips the wait outright
<id> wait_for_event
step <id>.expire on the timeout: read the row again; a terminal state
settles the node as answered
step <id>.resume resume(), close the span
⚠️ The second read is its own step, and <id>.resume is not it. The
approval wait pairs wait.check.n with wait.expire.n, and both are reads. A
timeout is not proof that the job did not finish: the event can be lost between
the emit and the waiter. <id>.expire runs on the timeout alone, so a wait the
event answered pays for no extra read.
The node declares nothing extra, and the event type is what selects the
reader. A small map keyed on the platform event type answers which row a correlation
names. It holds one entry. agentic.provider_job.completed.v1 reads
agent.provider_jobs by the job_id the match already carries.
mode event
event_type agentic.provider_job.completed.v1
match job_id: steps.submit.output.job_id
timeout_s 3600
on_timeout fail
A second field naming the same value is the same reference written twice. Two spellings of one fact disagree the first time an author edits one. An event type with no entry in the map registers its wait with no check step, which is every wait shipped before this. So no published definition changes and the validator gains no rule.
See asynchronous provider jobs.
A wait on an event that names no row is still open, and the timeout is its
backstop. An email reply is that case. The answer lives in an event that is already gone.
on_timeout = continue makes a missed reply cost a follow-up email, and not a
stalled Run. An author who cannot tolerate that models the
reply as a poll over a tool node instead.
A wait produces no output
steps.<id>.output exists for tool, agent and subworkflow. A wait joins the containers: a node that reads steps.<wait>.output is refused at publish, with the same message a sequence earns.
The event body is deliberately not a fact. A resumed workflow reads the reply through a tool node, which puts the read behind a span and inside the idempotency journal. Handed the event payload instead, the workflow would carry untrusted inbound content as a workflow fact with no span behind it, and a replay would read a payload that no longer matches the row. It is the same rule a branch obeys: a business fact is a step, never a hidden read.
The approval node
An approval node holds for a person. Like the wait node it needs one thing the platform must own at run time, so its shape is fixed here too.
approval node
id
action what a person is authorizing, one line
input the facts a person judges, the same {"$ref": ...} mapping every node carries
summary a static line a person reads, no interpolation
ttl_s required whole seconds, at least one
# the email sequence stops before it sends to a new account
action send the outreach email
input company: {"$ref": "steps.classify.output.company_name"}
subject: {"$ref": "steps.draft.output.subject"}
summary Review the draft before it sends.
ttl_s 172800 # 2 days
The TTL is whole seconds, and the key names the unit. It is the spelling timeout_s and max_run_duration_s already use, so an author reads one unit in every duration the platform takes. The SDK refuses a wait under one second and one that is not whole, so a fractional value is refused at publish rather than in a raise the Run cannot survive. The ceiling is the wait node's ceiling, 30 days, for the same reason: a longer hold outlives every Run that could carry it.
A ttl_s is required, for the same reason a wait timeout is. An approval raised by policy takes its expiry from the matching rule's approval_ttl. A node approval has no rule behind it, so nothing would set expires_at, and the Inngest wait timeout is computed from expires_at. A node with no TTL is therefore a Run that waits for ever, and validation refuses it.
summary is a static line, and input carries the facts. The declared shape said a template over input.* and steps.<id>.output.*. This page refuses an expression language for a condition and refuses a template for a reference, and a template in a summary is that same refused thing under a third name. It also has nowhere to go: agent.approvals stores preview, proposed_arguments and a NOT NULL arguments_hash, and a rendered string fills one of the three.
So the node takes an input mapping, resolved by the reference resolver every other node uses, and one static summary string. The row is then written the way an admission row is written.
| Node field | The agent.approvals column |
|---|---|
action | action |
summary | preview |
input, resolved | proposed_arguments |
input, resolved and hashed | arguments_hash |
A node approval carries a claim, and the memoized step is not enough. Inngest memoizes a step that returned. A raise step that dies after its insert commits is retried whole, and an unclaimed insert then files a second pending row: a person sees one gate twice, and the walk parks on the second row while waiting_ref_id names the first. The person answers the card they are shown, and the node still ends approval_expired.
uq_approvals_run_id_idempotency_key is partial on idempotency_key IS NOT NULL AND status = 'pending', so the key is what brings the row under that guard. The node writes <run_id>:<node_id>:<args_hash>, which is the segment journal shape, and a workflow node id is a step path. ApprovalService.create() then reads the first row back, and the retried step parks on the card a person already has.
The column still means "the claim an approved call takes" for the two policy checkpoints. A node takes no such claim, because it runs no call. It takes this one to be filed once.
An approval node produces no output. It joins wait and the containers: a node reading steps.<approval>.output is refused at publish. A person's decision is yes or no, and both are already the node outcome.
Four validation rules keep an approval node honest.
- A
ttl_sis required, whole, at least one, and at most the 30 day cap. - Every
inputreference resolves inside the allowed namespaces, which is the check every other node input gets. continue_on_erroris refused. The flag says the author expects this step to fail sometimes, and a tolerated gate lets the workflow proceed on a decision a person refused. That is the one thing the node exists to prevent.- A
parallelnode that holds awaitholds noapprovalin another child subtree. The reason is the wait node's own rule, read from the other side, and it is below.
A wait and an approval never share one parallel
Why two waits under one parallel are refused states the mechanism: resume() keeps a Run asleep by counting the pending rows in agent.approvals, and a wait node files no row. Two approvals are therefore safe, and two waits are not.
⚠️ One of each is not safe either, and the wait rule alone does not catch it. A parallel holding an approval in one subtree and a wait in another has one wait in one subtree, so it passes that rule.
parallel
├─ branch A approval a person answers it
└─ branch B wait(event) still parked
resume() counts 0 pending rows, so the Run reads `running`
branch B writes no heartbeat, and the reaper fails it `worker_lost`
That is the same ending the two-wait case has, reached from the other direction. The rule is therefore one rule over both node types, checked at publish: a parallel node whose child subtrees hold a wait holds no other parking node in any other subtree. A container of approvals alone stays unrestricted, because every one of them is a row resume() can count.
What a decision does to the node
Five endings, and one of them runs the next node.
| The row, when the wait ends | The node |
|---|---|
approved | ok, and the walk goes on |
rejected | failed, approval_rejected |
nobody answered by expires_at | failed, approval_expired |
cancelled | cancelled. RunManager.cancel() already wrote the Run |
| the row is gone | failed, approval_row_missing |
A rejection fails the node, and continue_on_error cannot tolerate it. A failed branch does not cancel its siblings, so the container settles at its last branch and the Run fails after it. That is the ordinary failed-branch path, and a gate needs no second one.
No authority is checked at the decision, and none is granted by it. ApprovalService.authorizes() tests the approver against the action as a scope, and a node action is an author's own words rather than a scope name. Nothing calls it here, because the node executes no call: it releases the walk, and every tool node after it runs its own checkpoints against the Run's principal. So a node approval is an acknowledgement gate. Human review already states the V1 rule this follows: visibility alone decides who may answer.
waiting_ref_id names the oldest unresolved approval, so the raise reads it rather than writing its own id. Two branches raise in two Inngest steps, in either order, and each one would otherwise leave the field naming itself. The raise marks the Run from oldest_pending(), which is the read resume() already makes, and the field is then right whichever step commits last.
The node writes the same agent.approvals row the two policy checkpoints write, with raised_by = node. See human review inbox.
Declared scopes
A workflow publishes the set of tool scopes it needs. Policy admission reads it, because a workflow node cannot ask the model for another way when a scope is missing.
required_scopes = the tool names on every tool node
∪ the required scopes of every referenced subworkflow
declared_scopes = required_scopes
∪ the tool names of every referenced agent
∪ the declared scopes of every referenced subworkflow
Two sets, because admission and the catalogue ask different questions.
| Set | Answers | Read by |
|---|---|---|
required_scopes | what fails if the principal lacks it | policy admission |
declared_scopes | everything this workflow could touch | the connections UI, and an admin reviewing a fork |
An agent node contributes to the second and not the first. A tool node cannot ask the model for another way, so a missing scope is fatal. An agent adapts wherever it runs, and refusing the workflow would deny a principal who can run that same agent alone.
Both sets reach through a subworkflow, and declared_scopes has to. PrincipalFactory mints a workflow Run's grant from declared_scopes, and a child Run's grant is the intersection of its own definition with its parent's. So a tool two levels down that the top-level set did not name is denied inside a Run that admission said was safe. A workflow that declares no tool_ids of its own is the reason the grant reads this field and not that one: read tool_ids for a workflow and the grant is empty, which the invoker reads as deny everything.
It is computed at publish and stored on published_config.
Both sets are derived, so publish removes them from the candidate before it validates it. The candidate is draft_config, and a fork copies a published_config that already carries them, so an author can hand a stale pair straight back. publish therefore drops the two keys, validates, recomputes them from the config it is about to write, and writes the recomputed pair. An author who hand-writes either key changes nothing.
And the two keys are why draft_config and published_config never compare equal. The surfaces page reads that comparison to show unpublished work, so a definition published a second ago would show as edited for ever. One exported constant names the derived keys, and every comparison drops them first. referenced_ids needs no such rule: it is a column and not a config key.
Revalidation validates a referrer. It never rewrites one, so the stored set does go stale, and one half of it is load-bearing for policy. Publishing agent A without a tool leaves workflow W's stored declared_scopes still naming it. Publishing subworkflow S with a new tool node leaves W's stored required_scopes missing it — and policy admission reads required_scopes, so W is admitted for a principal that lacks a scope W now needs, and the node fails deep inside a Run that admission said was safe.
So the recomputation is a validation rule, not a side effect: a referrer whose stored required_scopes or declared_scopes differs from the set recomputed against the candidate fails validation, and the publish is refused naming it. The admin republishes the referrer, which is the write that stores the corrected set. This keeps one property the page depends on everywhere else — publish establishes validity, and no Run recomputes it — and it costs one comparison inside a validation pass that already loads everything the comparison needs.
The regression rule above applies here too. A referrer whose stored set is already wrong against its own current config was broken before this publish, and it does not refuse one.
Validation
One validator entry point dispatches the kind specific checks internally.
| Kind | Checks |
|---|---|
| All | Schema, ownership, active references in the caller's own organization, publish authority, frozen size |
| Agent | Tool and skill existence, each referenced skill's tools are a subset of this agent's, model resolves, ContextPolicy sources, prompt and budget limits |
| Skill | Tool existence, size |
| Workflow | Unique node IDs across the whole workflow, references, target existence, type compatibility, no cycles, depth cap, branch shape, fan out budget, declared scopes |
Four of these checks already have code, and the validator calls it rather than writing a second copy. workflow_depth() in shared/ walks the cycle guard and the depth in one function. parse_workflow() reads the node tree and holds the unique ids, the reference namespaces, the container-output refusal, the branch default and the container nesting cap. validate_condition() reads a branch condition. ToolRegistry answers tool existence. Two implementations of one rule disagree once, and the disagreement is a definition that publishes and then fails mid Run. So the parser and the graph walk live in shared/, where the validator and the executor read one copy.
Validation is deterministic and model free. There is no validator plugin architecture in V1.
The skill validator does not check capability widening. It has no agent in hand. That check is on the agent, and the referrer rule reaches it from the other side. A table that lists it under Skill describes a check nobody can implement.
Publish authority is one scope, definition.publish, and it is granted by the role mapping exactly as a tool name is. The builder chat reaches publishing through a tool of that name, and the API route checks the same scope, so the two paths cannot diverge. See policy and governance.
⚠️ The scope gates every write to a definition, and not the publish alone. A definition is shared configuration, so a member who could rewrite an agent's draft could change what every colleague's next publish makes runnable. DefinitionService reads one answer through a seam, which is what stops the surface growing a second authorization path.
Phase 1 built the agent, skill and workflow validators, and that workflow validator covered six node types. Phase 2 added the wait and approval rules above, so it now covers all eight.
The fan out budget check has nothing to measure before Phase 4. The parser refuses any max_fanout or fanout_concurrency above one. Phase 4 accepts a wide tool step and keeps the width-one rule for an agent step.
ContextPolicy source names resolve in the process context registry. Publish checks every declared source, including a disabled source. It also checks each source's options against the strict schema that the registry accepted. An unknown source, an unknown option, a wrong option type, or one source name used twice fails publish. The Phase 3 process registry is empty until a source lands, so the production deploy still accepts only an empty source list.
Run snapshot
Before dispatch, SnapshotBuilder freezes the effective execution configuration.
Frozen in the Run snapshot
published definition config
rendered skill text
model facing tool contracts
ContextPolicy
model configuration
ceilings a child takes max_cost_cents from the root
Frozen in the Run, beside the snapshot
principal grant agent.runs.principal, its own column
Read live at each checkpoint
tool enabled or revoked state
policy rules
the actor's current rights, which can only narrow the frozen grant
credentials and connection status
handler implementation
business data and the fresh ContextBrief
Skills are rendered here, once. Execution reads the rendered text.
What the snapshot must hold
The executor reads fixed keys, so the builder writes those keys and no others. A snapshot that names max_run_duration where the reader looks for max_run_duration_s fails every Run of that definition with invalid_snapshot, and the definition validated cleanly. The two shapes are one contract.
an agent run a workflow run
model root
provider, model ceilings
temperature? the same five fields
max_output_tokens?
instructions read by parse_workflow() and _read_ceilings
the definition text plus
the rendered skills
tools[]
the model facing contracts
context_policy
ceilings
max_segments
max_agent_turns
max_tool_calls
max_run_duration_s seconds
max_cost_cents
A workflow definition declares budget_defaults too. A workflow run is a run, and its executor reads snapshot["ceilings"] exactly as the agent one does. One shape rather than two: a second ceiling type would be a second thing for the builder to fill in and a second thing for the meter to read.
A skill renders as a titled block under the definition text, in the order the agent names its skills. The order is part of the prompt, so it is the declared order and never a set: a republish that changed nothing an author wrote must build the same string, or a replayed segment changes the arguments hash of every tool call below it.
<the agent instructions>
## Skill: <name>
<description>
<instructions>
The builder refuses rather than drops. Every reason it can refuse was checked at publish, so a refusal at the freeze names a deploy or a seed that moved out from under a published definition: a tool the registry no longer holds, a skill that no longer resolves, or a config with no ceilings. Each one dropped instead would be a truncation of the kind this page forbids -- an agent that silently lost a tool it was published with, and no reader able to tell.
tools freezes the agent's own tool_ids and no skill's. A skill never widens an agent's tool set, so adding a referenced skill's tools here would grant at the freeze what the agent validator refuses at the publish. agent.runs.principal reads the same field, and the two must name one list.
budget_defaults uses the snapshot's own key names, max_run_duration_s included, so the builder copies the mapping and converts nothing. An earlier pass had the definition name a duration and the snapshot a count of seconds, which put one unit conversion between two JSON documents and no reader on either side of it. A definition holds JSON, and a person authors it through a builder, so nothing is gained by a second spelling.
freeze() takes the definition and nothing else. The principal is not in the snapshot, so a builder that took one would never read it, and publish must build the same snapshot with no principal in hand. One signature therefore serves both callers: publish measures what it is about to make runnable, and the run freezes it. Two functions computing one value disagree the first time either changes, and the disagreement is a definition that passes the size check and then fails every start.
The principal grant is frozen, and it is not in the snapshot. agent.runs carries principal as its own column, and PrincipalFactory.for_run() is its one writer. Listing it inside the snapshot as well would give one fact two homes, and the two disagree the first time either writer changes. The snapshot holds execution configuration; the column holds authority.
The snapshot is why V1 needs no revision history to keep in flight work stable. It is also why an emergency tool revocation or policy change still takes effect at once.
Snapshot storage
V1 stores the snapshot inline, on the Run row. This is the decision, not a placeholder. One row holds everything a fresh worker needs, and a resume reads one row.
Three guards keep it affordable.
- A Run list query never selects
snapshot. The repository exposes a narrow list projection and a separateget_snapshot(). A list endpoint that selects the whole row pulls megabytes of frozen tool contracts to render a status column. - A size check at publish. The frozen size is deterministic, so validation computes it and refuses a definition whose complete snapshot passes 256 KiB.
- A width trip-wire. Track the widest live fan out. If more than 50 live child Runs share one definition, move to the content addressed table below. This one is an operations metric and not a check in the write path. Nothing refuses a publish or a start on it, and no code in the definitions package reads it.
The snapshot is checked, and never truncated
bound() is the one payload boundary, and it works by dropping whole top level items. That is right for a RunResult, a ToolResult and a span. It is wrong for a snapshot: the dropped item is the tool contracts or the rendered skill text, and the Run then executes an agent that silently lost a tool it was published with. No error is raised and no reader can tell.
So the snapshot is not a bound() caller.
at publish the complete frozen size is computed and refused above 256 KiB
at freeze an oversized snapshot fails the start with 'snapshot_too_large'
at freeze a contract the builder cannot assemble fails the start with
'snapshot_unbuildable'
never an item is dropped from a snapshot
A refusal at the freeze arrives as an outcome, never as an exception. The builder refuses a tool the registry no longer holds, a skill that no longer resolves, and a config with no ceilings. RunManager.start() answers a closed set and four callers branch on it, one of them an Inngest step: an exception left to escape leaves that step, the function retries it to exhaustion, and every sibling branch of an enclosing parallel node ends with it. snapshot_unbuildable maps to 409, because the request is well formed and a retry cannot help until someone fixes the deploy.
The check belongs at publish because the size is a pure function of published_config, the rendered skills and the registry contracts. An admin then learns it at the write they made, rather than at the first Run. The start time failure is the backstop for the two inputs that move between the two moments: a tool contract that grew, and a skill that was republished with longer text. The skill is the likelier of the two, because the rule directly above says a skill publish reaches a referring agent that never republished.
This replaces the earlier 64 KB size trip-wire, which could not fire: bound() had already cut the payload to 32 KB, so a p95 above 64 KB was unreachable. See the payload boundary.
The width trip-wire matters as much, and it is easy to miss. Size measures one row. A batch measures the count. An email sequence over 500 people creates 500 child Runs of one agent definition, and each one freezes the same tool contracts and the same rendered skill text. Every row is small, so the size trip-wire never fires, and the storage is still 500 copies of one blob.
The deduplication table is a lookup, not a revision system. Every Run of the same published definition shares one row, and no other contract changes. Do not build it before a trip-wire fires. Repeated identical blobs are cheap to compress and expensive to design around early.
Rules
- One repository shape serves agents, workflows and skills.
- One validator entry point. The kind specific checks stay internal.
- Publish establishes validity. Do not revalidate a definition on every Run.
- Publish revalidates the definitions that reference the published one, and only a regression refuses it.
- A referrer whose stored
required_scopesordeclared_scopesno longer match the recomputed set fails validation. - Publish validates the candidate config before it writes. There is no transaction to roll back.
expected_updated_atguards a draft save and a publish, and neitherdisable,enablenordelete_draft. A stale write returnsstale, and never raises.expected_updated_atis an opaque string end to end. A millisecond-precision client that re-formats it makes every write answerstale.- The
+of the token's offset travels as%2B. An unencoded one reads as a space and every write answersstale. - A draft patch replaces a named field. It never merges into one.
DefinitionServiceowns the lifecycle and the authorization.- The refusal order is
not_found,forbidden,not_published,invalid,referrer_limit. update_draftruns the candidate validation alone. A draft is in no referrer set.- One publish holds one loader with one cache, and every validation pass shares it.
- The API and the UI never write lifecycle fields directly.
- No cascading disable.
disableis reversible, andenablerevalidates the config as stored. - A refusal at the freeze answers
snapshot_unbuildable. The start never raises. - A
disabledreferrer counts against the cap and never refuses a publish.enableis where its break surfaces. - Every repository method takes an
organization_id. Service-role reads apply no RLS. - A custom definition references its own organization only. A platform template is forked before it is referenced.
- Publish writes
published_atbesidepublished_config. The two are one constraint, and every publish moves it. - Cycle detection runs before the depth function, which has no cycle guard. It walks
subworkflowtargets only. - An indirect referrer is not revalidated. The run time depth re-check is the only mechanism that sees it.
- A draft is the only row that deletes. A published definition is disabled.
- A fork source is
active. A draft and a disabled row both answernot_found. - A fork of a platform template is a deep fork. It copies the reachable set and rewrites every id.
- The deep walk crosses the organization boundary only. A reference already in this organization is shared, never copied.
- Every deep fork mints a fresh set, and it writes the leaves before the root.
- Disable refuses a new Run tree. It never refuses a child of a running tree.
- The Run snapshot is stored inline. A list query never selects it.
freeze()takes the definition alone. Publish and the run start call one function.- The snapshot writes the keys the executor reads,
ceilings.max_run_duration_sin seconds among them. - A workflow definition declares
budget_defaults, because a workflow run spends the same five bounds. - The builder refuses a tool or a skill it cannot assemble. It never drops one.
- The snapshot is size checked at publish. It is never truncated, and it is not a
bound()caller. - The principal grant lives on
agent.runs.principal, and not in the snapshot. - A node ID is unique across the whole workflow. A container produces no output of its own.
- A branch declares a
default, and its conditions read onlyinput.*andsteps.<id>.output.*. - Depth counts workflows, the cap is three, and one function serves publish and run time.
- A wait node always declares
timeout_s, and an approval node always declaresttl_s. Both are whole seconds, capped at 30 days. - A
parallelnode parks in at most one of its child subtrees. Awaitbeside anapprovalis refused for the same reason two waits are. - An approval node takes
inputand a staticsummary. There is no template anywhere in the language. - An approval node refuses
continue_on_error. A tolerated gate is no gate. - A rejected or expired approval fails its node. An approval node produces no output.
- A tool fan out declares
max_fanoutandfanout_concurrency, and its worst case wall clock fits the step budget. - Phase 4 permits a wide tool only when it is workflow allowed and read only.
MAX_FANOUTis 200,MAX_FANOUT_CONCURRENCYis 20, and both fields default to one.FANOUT_STEP_HEADROOM_Sreserves 10 seconds outside the item batches.- A wide tool's declared result data fits the 3 MiB fan-out data budget.
- All declared leaf output data fits the 24 MiB workflow state budget.
- Publish returns
fanout_not_availableuntil the wide executor is present. - A wide tool input is one reference to an ordered list of complete argument mappings.
- An agent fan out stays at width one until the runtime can start and join many child Runs.
- Every
ContextPolicysource resolves in the process context registry, and its options pass the source's strict schema. - One
ContextPolicynames each source once, including disabled sources. - A run time list longer than
max_fanoutfails the node. It is never truncated. - A retry covers transient faults, and the function owns the count. It cannot repeat a completed effect.
- A reference to a step that did not run fails the node, and never resolves to null.
- A publish revalidates direct referrers only, and refuses to become the 51st referrer of anything. The count excludes the publisher and covers the targets this publish adds.
- The referrer set is
published_configonly, and it is read fromreferenced_ids, an indexed column that holds no tool id and no platform id. - A seeded platform template carries a seeded
referenced_ids, becausepublishnever writes one. - A workflow publishes
required_scopesanddeclared_scopes. Admission reads the first, and the Run's principal grant reads the second. - Both scope sets reach through a subworkflow. A tool two levels down is named at the top, or it is denied at run time.
- Both scope sets are derived. Publish drops them from the candidate, recomputes them, and writes the recomputed pair.
- The draft-versus-published comparison drops the derived keys, or a published definition reads as edited for ever.
- No revision, history or rollback subsystem for a tenant-authored definition in V1. Managed capability executors carry released revisions and an operator rollback; see "Upgrade and rollback".
Minimum contract tests
- A stale
expected_updated_atfails cleanly on a draft save and on a publish. - A publish of a never-published draft writes
published_at, and the row satisfiesdefinitions_published_at_shape. - An
expected_updated_atre-formatted at microsecond precision still matches the row, and one truncated to milliseconds matches nothing. - The API returns
expected_updated_atas a string, and no surface parses it into a date type. - A fork lands the source config in
draft_config, and the new row satisfiesdefinitions_published_shape. - A publish is not refused by a
disabledreferrer that would fail against it. enableon that referrer fails, and names the definition that broke it.- A disabled referrer still counts toward the referrer cap.
- A referrer lookup for a platform template returns nothing, because a platform template is forked before it is referenced.
- A publish naming a definition of another organization fails validation.
- A fork of a draft is refused.
- A workflow that references itself fails with a cycle error, and never a
RecursionError. - A three-workflow chain whose deepest link is published last publishes, and the parent's fourth level is caught by the run time depth re-check and not by the publish.
- An agent naming a provider and model pair the registry does not hold fails validation.
- A draft patch that sends a shorter
tool_idsremoves the tools it omits. - An invalid tool or skill reference blocks an agent publish.
- An agent naming a model the registry does not hold fails validation.
- A skill cannot grant a tool the agent does not have.
- A workflow cycle, depth or type error blocks the publish.
- A referenced active definition cannot be disabled.
enablereturns a disabled definition toactive, and revalidates it first.enablefinds a stored scope pair that went stale while the definition was off.enablefails when a definition it references was disabled while it was off.delete_draftremoves a draft, and refuses an active or disabled definition.- A fork of another organization's custom definition is refused.
- A disabled definition refuses a new Run tree, and allows a child of a running tree.
- Publishing an agent that drops a tool fails when a published workflow needs that tool.
- Publishing a subworkflow that breaks the depth cap of a referring workflow fails.
- A Run snapshot stays stable after a later draft edit and publish.
- The Run list projection does not read the snapshot column.
- A wait node with no
timeout_sfails validation, and so does an approval node with nottl_s. - A
parallelnode with awaitin two child subtrees fails validation, and so does one with awaitbeside anapproval. - An approval node declaring
continue_on_errorfails validation, and so does a node readingsteps.<approval>.output. - An approval node whose
ttl_spasses the Run deadline expires at the deadline, and no later. - Two branch approvals of one
parallelnode resolve in either order, and the Run readswaitinguntil the second one is answered. - A wait node whose
timeout_spasses the Run deadline parks until the deadline, and no longer. - A fan out whose worst case wall clock passes the step budget fails validation.
- A fan out whose item batches leave less than 10 seconds of step headroom fails validation.
- A fan out concurrency above its maximum width fails validation.
- A fan out above either platform cap fails validation.
- A boolean, string or decimal fan-out limit fails validation.
- A write, send or workflow-refused tool cannot publish at width above one.
- A fan out whose declared result data passes 3 MiB fails validation.
- A workflow whose declared leaf result data passes 24 MiB fails validation.
- An otherwise valid wide tool returns
fanout_not_availablewhile the execution gate is false. - A wide tool input that is not one whole reference fails validation.
- A wide agent node fails validation in Phase 4.
- A retried tool node with unchanged arguments produces one vendor effect.
- A node reading the output of an untaken branch fails with
unresolved_reference. - A width of 200 and concurrency of 20 validates against ten tool timeouts, when both its time and data budgets fit.
- A publish that would become the 51st direct referrer of one definition is refused.
- A re-publish of a definition already among a target's 50 referrers is allowed, because it grows no count.
- A publish whose referrer validation fails writes nothing, and the referrer is named.
- A referrer already invalid against its own published config does not refuse an unrelated publish.
- Publishing an agent that drops a tool fails when a referring workflow's stored
declared_scopesstill names it. - A draft that references a definition is not counted as a referrer, and is not revalidated.
- A branch with no matching case and no
defaultfails validation. - A reference to a
sequence,parallelorbranchoutput fails validation. - Two nodes in different containers cannot share one node ID.
- A subworkflow chain three workflows deep publishes, and four deep is refused.
- A fork of a
disableddefinition is refused, and answersnot_found. - A fork of a platform workflow that names a platform agent lands two drafts, and the copy names the copy.
- A fork of the caller's own definition copies one row, and its references still name the original rows.
- The same platform template forked twice lands two independent sets, sharing no row.
- A fork whose reachable set holds a definition that does not resolve is refused, and names it.
- A deep fork writes every leaf before the root, so no draft ever names an id that does not exist.
- An
expected_updated_atwhose offset carries a+matches the row, against the live stack. - A publish drops
required_scopesanddeclared_scopesfrom the candidate, and writes the recomputed pair. - A definition published a moment ago compares equal on
draft_configandpublished_config, the derived keys dropped. freeze()writesceilings.max_run_duration_sin seconds, andAgentExecutorreads the snapshot it wrote.parse_workflowreads the snapshot the builder wrote for a workflow.- A tool the registry no longer holds refuses the freeze, and no snapshot is short a contract.
- A workflow with no
budget_defaultsfails validation. - Publish answers
forbiddenfor a non-admin before it answers any state outcome, andnot_foundbefore that. - An agent naming a context source the process registry does not hold fails validation.
- An agent naming a registered context source with valid options passes validation, enabled or disabled.
- An agent naming one context source twice, an unknown option, or an option with the wrong type fails validation.
disableandenableon a draft answernot_published, and a repeateddisableanswersok.- A run time list longer than
max_fanoutfails the node withfanout_exceeded. - A snapshot over the size limit fails the publish, and no snapshot is ever truncated.
- The frozen snapshot holds no principal grant;
agent.runs.principaldoes. - Publishing a skill changes the rendered text a new Run of a referring agent freezes.
- A wait node whose
matchnames an unreachable value fails validation. - A node reading
steps.<wait>.outputfails validation, exactly as one reading a container does. - A workflow's
required_scopescover every tool node and every referenced subworkflow, and no agent node. - A workflow's
declared_scopesalso cover every referenced agent's tools, and every referenced subworkflow's declared set. - A workflow Run's principal grant is its
declared_scopes, so a tool node of a published workflow is never denied. - A skill is size checked at publish, and refused at the freeze of a Run.