Tools and integrations

A tool is one atomic capability an agent may call. It is the only way an agent changes anything.

The model proposes. ToolInvoker decides, executes and bounds the answer.

Agno agent
   -> AgnoToolAdapter declaration
   -> proposed tool call
   -> ToolInvoker
        -> live tool state
        -> input schema
        -> journal read (write and send)
        -> policy action checkpoint (stage 1 grant, stage 2 rules)
        -> idempotency claim (write and send)
        -> connection + handler
        -> result boundary
        -> span + usage
   -> ToolResult
   -> Agno agent continues

Tool, skill, service

A skill teaches. A tool acts. Never expose a raw REST route, a generic api_request, a repository or SQL to a model.

Four checks

Do not collapse the four. Visibility is not authorization.

For public service-key reads, scoped_db(organization_id) is the tenant guard. Agent-schema reads filter organization_id explicitly. RLS remains a guard for user-key paths, but it does not filter the API service key.

Core code

services/tools/
  models.py             ToolSpec, ToolResult, ToolError,
                        NAME_PATTERN, model_tool_name
  registry.py           ToolRegistry, build_registry
  declarations.py       code declared specs + MCP discovered specs
  invoker.py            ToolInvocation, ToolInvoker, ApprovalRequired
  default_invoker.py    policy, approval, accrual, idempotency and execution order
  output_validation.py  output normalization and JSON Schema validation
  result_boundary.py    redaction, bounding, untrusted content envelope
  handlers/
    base.py             ToolHandler protocol
    crm/
      common.py         shared CRM handler helpers
      companies.py      company reads, updates and canonical upsert
      people.py         people search and canonical upsert
      lists.py          idempotent list membership
    memory.py           durable observation and preference memory
    research.py         Exa, Parallel, Firecrawl

The connection resolver, Nylas email handler and generic MCP handler are planned extensions. They are not modules in agentic-platform yet.

AgnoToolAdapter is not in this package. It lives beside the runtime it serves, at runtime/agent/agno/adapter.py, because it declares an Agno callable and raises an Agno exception. Keeping it here would put Agno imports in two packages, and the import contract could then name neither of them. Nothing else in this page moves: the registry, the invoker, the policy and the handlers stay framework neutral. See the Agno boundary.

ToolSpec

ToolSpec is the framework neutral contract. AgnoToolAdapter shows the model the description and the input_schema, under a derived name.

class ToolSpec(BaseModel):
    name: str                       # the platform name: domain.verb, globally unique and stable
    description: str                # the model reads this
    input_schema: dict              # JSON Schema
    output_schema: dict             # JSON Schema
    version: int = 1                # bumped whenever a schema changes
    schema_hash: str                # of input_schema + output_schema; the resync compares it
    kind: Literal['internal', 'vendor', 'mcp']
    binding: HandlerBinding | RemoteBinding
    side_effects: Literal['read', 'write', 'send']
    connection: str | None = None   # provider key the ConnectionResolver must satisfy
    timeout_s: int = 30
    retry: Literal['none', 'safe'] = 'none'
    metered: bool = False           # the handler writes a usage row
    repeatable: bool | None = None  # required and false for write/send; absent for read
    approval_preview: bool = False  # the handler renders the approval summary
    policy_facts: list[str] = []    # target facts a rule may read about this call
    batch_argument: str | None = None
    max_batch_size: int | None = None
    max_results: int | None = None
    max_output_bytes: int | None = None
    item_kind: str | None = None    # the ResourceRef kind of one item of `data`

Seven fields deserve an explanation.

• name is the platform name, and the model never sees it. OpenAI and Anthropic both accept a letter, a digit, an underscore and a hyphen alone in a function name, so either provider refuses email.send on the first model call. OpenAI documents a cap of 64 characters, and the platform takes the lowest cap a provider documents. AgnoToolAdapter derives the model facing name when it declares the tool, and the adapter owns that map alone. A read call carries the platform name in its own body. A paused write or send reaches the drain loop under the model facing name, and the adapter maps that one back. Every one of these keeps the dot: Principal.scopes, declared_scopes, every published definition, every frozen snapshot, every policy rule, every idempotency scope and every live run event.
• input_schema is JSON Schema, not a Python type. An internal or vendor tool declares a Pydantic model in code, and registration generates the schema from it. An MCP tool carries the schema its remote server published. One field then serves both, and the catalogue row stays storable.
• binding never carries code, and it is not a string.
  PYTHONCopy

  class HandlerBinding(BaseModel):
      handler_key: str          # resolves in the trusted handler registry
  
  class RemoteBinding(BaseModel):
      connection_id: UUID
      remote_name: str          # the tool name the remote server published
  

  
  A single '<connection_id>:<remote_name>' string needs a parser, and a parser gets written twice and disagrees with itself once. A database row still cannot introduce executable Python: handler_key resolves in a registry the deploy owns.
• retry is safe only for a read. A write or a send retries through Inngest and the idempotency claim, never inside the handler.
• policy_facts is how a rule asks a question the arguments cannot answer. A rule such as email.send AND recipient_is_new needs a CRM lookup. Policy may not query domain data, and the handler runs after the decision, so the tool declares the fact and the invoker resolves it first. See policy and governance.
• batch_argument names the public list that max_batch_size bounds. The two fields appear together or not at all. The name must resolve to a top-level array in input_schema. An optional batch that is absent counts as zero items. The invoker does not guess from every list in the argument tree. A filter list is not necessarily a batch, and two lists can form a Cartesian product.
• item_kind is what lets the invoker turn a Dropped marker into a ResourceRef. bound() writes a marker and never a ref, because a pure function has no row to name. The invoker does hold the rows, and it is the one caller that substitutes — but a marker carries a size and a reason, not a kind. So the tool declares the kind of one item of its own data, such as crm.person, and the invoker reads the id off the item it replaced. A tool whose data is not a collection of rows leaves the field unset, and its markers stay markers.

There is no scopes field. V1 has one scope vocabulary: the platform name. Principal.scopes holds the tool names the run may call, and it is already the intersection of the agent grant and the caller rights. A second vocabulary adds a mapping table and answers no question the tool name does not answer.

V1 needs no tool revision history. A run freezes the contract it showed the model, and reads the live enabled state, the policy, the credentials and the handler at each call.

Where a spec comes from

Two sources, one catalogue.

internal_tool() and vendor_tool() generate schemas from the same Pydantic models and create the same HandlerBinding. The internal factory sets kind='internal'. The vendor factory sets kind='vendor'. The vendor factory does not infer a connection, metering, retries, or limits. A caller declares each supported field explicitly. The MCP resync builds its own remote declaration when that binding lands.

An MCP tool is named mcp.<connection_slug>.<remote_name>. The prefix keeps a remote server from claiming a platform name. Registration rejects a platform name collision instead of shadowing.

An MCP server may drop or change a tool between runs. The invoker therefore checks that the bound remote tool still exists before it executes, and returns tool_unavailable when it does not.

Schema drift is a version check, not a presence check

Presence is not enough. A server that keeps a tool and changes its input schema accepts the call and rejects the arguments, so a run fails with invalid_input on a call that was valid when the snapshot froze. The model is shown the frozen contract and cannot know.

Every spec carries a schema_hash and a version, and the resync compares the first to decide whether to bump the second. Both are fields on ToolSpec above, because the check reads them from a frozen snapshot and from the live registry, and a value that lives in neither cannot be compared.

on connect and on resync
  hash the remote input_schema and output_schema
  unchanged  -> nothing happens
  changed    -> write the new spec, and bump ToolSpec.version

at invoke time
  snapshot.version == registry.version   -> execute
  differ                                 -> ToolResult error 'tool_changed'

tool_changed is a stable error code and a value, not a raise, so the agent reads it and adapts exactly as it does for denied. A workflow tool node fails its node, and continue_on_error decides the rest.

The resync runs hourly per connection, and on every connect. An hour is the exposure window, and tool_changed closes it at the moment of the call rather than at the moment of the sync.

This answers what was open decision 1: a changed schema neither disables the tool nor forks it. The new spec wins for a new run, and a run holding the old contract stops cleanly instead of sending wrong arguments.

ToolRegistry

The registry answers what a tool is.

class ToolRegistry:
    async def get(self, name: str) -> ToolSpec: ...
    async def resolve(self, names: list[str]) -> list[ToolSpec]: ...
    async def list_for_organization(self, organization_id: UUID) -> list[ToolSpec]: ...

The registry answers catalogue resolution and visibility. It never answers per call permission.

The full catalogue never reaches a model. An agent sees the tools its definition names, and nothing else.

There is no ToolFactory, and there was one

The factory closed each spec over ToolInvoker and answered a BoundTool list, and AgentExecutor called it once per segment. Nothing calls it now, because two later changes took both of its jobs.

The snapshot took the visibility half. A definition names its tools, publication freezes those contracts, and the run reads them as AgentExecutionRequest.tools. A model therefore sees exactly the frozen list, and a factory filtering the same list a second time answers the same list.

The adapter took the calling half. AgnoToolAdapter turns one frozen contract into one framework callable, and ToolInvocation carries the ambient identity a closure used to hold. A BoundTool between them is a third shape for one fact.

So visibility is the snapshot, and authority is stage 1 of the policy engine. Two mechanisms, in two layers, and neither restates the other.

⚠️ ToolInvoker holds no scope check of its own. Stage 1 reads the grant, live, at the action checkpoint. A frozen read in the invoker would pass a tool the live read stripped, and it would refuse before the engine ran, so a scope refusal would reach no agent.policy_decisions row.

⚠️ principal.scopes must therefore hold the definition's tool names. The principal is the intersection of the agent grant and the caller rights, and stage 1 refuses a call whose name it does not find. A principal minted with an empty scope list refuses every call a run makes, and the refusal reads as a policy decision rather than as an unfilled field. The same set is what a child run intersects against, so an empty parent grant empties every grant below it too. See runtime execution.

ToolInvocation and handlers

@dataclass(frozen=True)
class ToolInvocation:
    """The ambient identity of one call. It is not model context."""
    organization_id: UUID
    user_id: UUID | None
    run_id: UUID
    definition_id: UUID
    step_path: str                  # workflow node ID, or the literal 'agent'
    principal: Principal
    ceilings: RunCeilings           # the REMAINDER for this segment, not the frozen budget
    deadline: datetime
    source: RunSourceKind

class ToolHandler(Protocol):
    async def execute(self, inv: ToolInvocation, args: BaseModel) -> ToolResult: ...

class ApprovalPreviewHandler(Protocol):
    async def preview(self, inv: ToolInvocation, args: BaseModel) -> str: ...

The model supplies the public arguments only. The organization, user, run, definition, principal, source and every credential are runtime owned.

There is no span_id field, and there was one. One invocation is built once per segment and shared by every call in it, so an invocation is older than every call made through it: one span_id would name one span for thirty tool calls. Span identity is ambient for exactly this reason, read through current_span_id(), and ToolInvoker opens the tool span itself as a child of whatever is current. Two mechanisms for one fact disagree the first time either moves, and this pair disagreed on its first call. See observability.

ceilings carries the remainder for four bounds, and the frozen budget for the fifth. AgentExecutor subtracts the turns, the tool calls and the wall clock a run already spent, so a metered call reads what is left of each. It subtracts nothing from max_cost_cents, because the meter answers the spend of the whole tree and a total compares against a total. Subtract there as well and the run stops at half its budget.

preview() is optional. A tool that sets approval_preview renders the line a person reads in the approval inbox, because only the handler knows that email.send means "one message to 3 new recipients". Without it the inbox falls back to a generic argument render.

preview() must be read only, and it runs on a call nobody has allowed yet. Policy has just answered require_approval, so the action is explicitly not authorized, and the invoker calls the handler anyway to render the line. Five rules keep that safe.

• It performs no write, no send, and no metered call. A preview that spent money would spend it on a request a person then rejects.
• It reads under the same principal, so it can see nothing the run could not.
• It runs under the shorter of timeout_s and the run deadline. A slow summary cannot hold the approval path without a bound.
• It returns a string. The invoker trims it and replaces each whitespace run with one space, so the stored value is one non-empty line.
• The normalized line is at most 512 UTF-8 bytes. A non-string, an empty line, an oversized line, a timeout, or an exception stores preview=None. Human Review then keeps its argument view. The invoker logs the fallback, and the approval still exists.

Vendor and MCP output is untrusted input.

An internal handler is not protected by RLS, and calling it that is the dangerous shorthand. ac-python-api reads public on the service key, which bypasses every policy, so the tenancy guard is scoped_db(organization_id) applying an explicit filter against ORG_SCOPED_TABLES. A handler that reaches for the admin client directly compiles, passes review and reads every tenant.

Global Intelligence is the narrow exception. public.intel_* has no organization_id, is absent from ORG_SCOPED_TABLES, and denies every user role through RLS. Its dedicated domain stores own the service-role client. An Intelligence handler calls those stores and never imports the admin domain or the client. A test must prove that the write does not include organization_id.

⚠️ The guard is scoped_db, not "go through the domain service". Both CRM services reach the legacy agent stack:

crm.companies.service -> crm.activities.service
                      -> domains.envoy.sequences.steps.step_orchestrator
                      -> domains.envoy.sequences.service
                      -> workflow_engine.services.workflow_execution_service

src.agentic never imports the legacy agent stack is an import-linter contract. The legacy stack stays a working fallback until cutover. Importing either service breaks the contract and pulls agno into a second package. crm.people.service reaches the same edge.

So an internal handler takes scoped_db directly, exactly as crm.search.service already does, and never the admin client. It does not import a legacy domain service or repository.

A platform-local capability service under src.agentic can share write rules with a platform surface. The service takes the same scoped_db client and does not cross into the legacy stack. Do not add this layer when the handler is the only caller.

The handler or its platform-local service carries what the legacy service was carrying: the soft delete filter and the column projection. A tool must not answer a row the product hides. Both rules belong in a test, because neither rule is visible in a review of the query alone.

ToolInvoker

class ToolInvoker:
    async def invoke(self, tool_name: str, args: dict, inv: ToolInvocation) -> ToolResult: ...

The path is fixed.

open the `tool` span, as a child of current_span_id()
 -> resolve the snapshotted contract
 -> check the live enabled state, the remote tool presence, and the spec version
 -> validate the input schema and max_batch_size
 -> read the journal when write or send
      absent     -> continue
      completed  -> return the stored result, and tag the span replayed
      processing -> raise ClaimInFlight; another worker holds a live lease
      conflict   -> ToolResult error 'conflict'
 -> read the authorized_by the runtime handed this segment
      covers the call, approved -> the action checkpoint is already decided
      covers the call, rejected -> ToolResult error 'rejected'
      names another proposal    -> ignored; the call is decided live
 -> resolve the policy_facts a matching rule needs
 -> policy action checkpoint
      stage 1 reads the grant, live, so a right stripped mid run is gone here
      fault            -> ToolResult error '<fault_code>', stop 'fault'
      deny             -> ToolResult error 'denied'
      require_approval -> the claim already holds a rejection of this call
                            -> ToolResult error 'rejected'
                          otherwise
                            -> persist the approval, raise ApprovalRequired
      allow            -> continue
 -> accrual checkpoint, when the tool is metered
      deny             -> ToolResult error 'budget_exhausted', stop 'budget_exhausted',
                          meta.partial_reason 'budget_exhausted'
      fault            -> ToolResult error <the fault code>, stop 'fault'
 -> claim the idempotency key when write or send
      claimed    -> continue
      completed  -> return the stored result, and tag the span replayed
      processing -> raise ClaimInFlight; another worker holds a live lease
      conflict   -> ToolResult error 'conflict'
 -> resolve the connection, then the handler
 -> execute under timeout_s and the run deadline
 -> validate the output schema
 -> redact, bound and tag the result
 -> complete the claim, or release it when the failure is retryable
 -> close the span, link the usage row
 -> ToolResult

Only ToolInvoker calls a handler from an execution path. Nothing else does.

The span opens first, so a refused call is still in the tree. denied, rejected, tool_changed, invalid_input, conflict and budget_exhausted all return before the handler runs, and each one is a thing the model tried to do. A span opened after the checkpoints would record the calls that succeeded and hide the calls that policy stopped, which is the opposite of what an auditor opens the tree for. The refusal closes the span with status = 'error' and its stable code.

The journal is read before the decision, and claimed after it

The journal read and the claim are two steps, and the decision sits between them. A design that read the journal inside the claim would put the whole answer after the policy checkpoint. A turn that holds two gated calls would then never finish.

requirement 1  email.send(to='jo@acme.test')    gated
requirement 2  email.send(to='sam@other.test')  gated

pass 1    call 1 raises. Nothing ran
resume A  call 1 is approved and runs. Call 2 raises
resume B  approval B names call 2, so call 1 reaches the invoker unauthorized

At resume B the journal answers completed for call 1. The pass moves on and decides call 2.

Read the journal after the checkpoint instead, and policy answers require_approval for call 1 a second time. The inbox then shows a person a message the platform already sent. Every resume repeats that, and the run ends on the tool call ceiling.

A decision about an effect that already happened is not a decision. Policy cannot unsend the message. The platform already recorded the answer, so the journal answers first.

The claim itself stays after the checkpoints. A call that stops for a person takes no claim, and the claim it would have taken is the one the resume needs.

⚠️ The read can go stale, so the claim decides again. The checkpoints take time, and a human round trip takes days. Another attempt of the same segment can complete the very key this one read as absent. The claim is therefore the authority.

The two steps answer four outcomes each, and the four are not the same four. Only read() answers absent, and only claim() answers claimed. One shared branch helper gives the claim path an absent -> continue arm. A claim that raced and lost then reaches the handler, and that is the second message this step exists to stop.

read()    absent   completed   processing   conflict
claim()   claimed  completed   processing   conflict

The key is derived once, before the read. Both steps use that one value. V1 refuses repeatable = True, so no ordinal can make the read and claim name different keys.

⚠️ A replay is answered before the checkpoint, and policy is not asked. The effect already happened, so policy is never asked to decide it again. Only a write or a send journals, and the key names one organization, one run, one step and one arguments hash, so a replay is always one run reading back its own effect. The first call this run has not already made meets the live grant and the current rules.

A metered call checks the budget

AgentExecutor checks accrual before each segment, and WorkflowStepExecutor before each node. That is enough while a node costs about the same as a model turn. It is not enough for a fan out.

Signals Search searches people across every company that survived pruning, inside one step. Between the accrual check at the top of that node and the check at the top of the next one, the run can make hundreds of paid vendor calls. The ceiling is read once, and the overshoot is the whole fan out.

So a tool that declares metered passes the accrual checkpoint before it executes.

ToolSpec.metered = True   -> the invoker calls accrual first
ToolSpec.metered = False  -> no extra read; the node boundary is enough

This adds no fourth checkpoint. Accrual keeps one definition and gains a second caller, exactly as the action checkpoint has one definition and many callers.

A metered call reads the run tree only. The organization day ceiling is checked at the segment and node boundaries, and not on every paid call. A run cannot pass the organization ceiling faster than it passes its own, so the second read buys nothing and costs a whole-day aggregate per call inside a fan out.

before a metered tool call     the run tree total          indexed on root_run_id, one run's rows
before a segment or a node     the run tree AND the day    rare, so the cost does not matter

Both read ai_usage_log, the canonical meter. Neither reads ai_usage_daily: a rollup lags, so a run's own in flight spend is missing from it and the ceiling would never fire. See policy and governance.

A run stopped this way still succeeds with a partial reason. The invoker returns budget_exhausted as a value, so the agent or the step can finish with the items it already has. See policy and governance.

A metered call the meter cannot see

Two things stop the checkpoint before it reads anything. No accrual checker is wired, or the call runs outside a run scope. Both are wiring faults, and both answer internal_error.

They never answer budget_exhausted. The money code sends an operator to ai_usage_log and to the run ceilings, and both show headroom.

Sentry receives the fault one time. The log receives every call. The fault runs on the path of one tool call. A fan out node makes hundreds of calls inside one step. A report per call gives the Sentry issue one repeated message. That message then hides every other defect. The invoker holds no durable step, so it holds a set of the faults it reported.

WorkflowStepExecutor bounds the same shape by a different amount. Its durable step reports one time for each step execution, so a later node and a later run report again. The invoker's set reports one time for the life of the invoker.

The key names the tool and the cause. It holds no run id. Both causes are static. A missing checker is a defect of the deploy. A missing run scope is a defect of the call site. Neither one varies by run, so a run id in the key gives one report for each run of a fan out.

⚠️ An operator who resolves the Sentry issue receives no second event. That invoker stays silent for its life. Both causes change only on a deploy, and a deploy builds a new invoker with an empty set. The log carries every call until then, and a retry in another worker process gives one more report.

The refusal carries its own stop, beside the code

ToolInvoker answers a refusal it made and a tool that failed as the same type. The two need different answers, and the code cannot separate them. A remote tool may answer budget_exhausted about a vendor budget. A caller that read the code would turn a run that really failed into a partial success.

So ToolResult carries the field stop. Only DefaultToolInvoker writes it, _bound drops a value a handler wrote, and model_payload never renders it.

ToolStop = Literal['budget_exhausted', 'fault']

budget_exhausted   the run ends partial under that reason; the work done stays done
fault              the run ends failed under the result's own error code
None               an answer the model adapts to, or a success

Six refusals on the metered path set fault. Each one is a defect of the deploy or of the call site, and no model can act on any of them. Other internal_error refusals outside that path still set nothing. PLATFORM_STOP_CODES stops a workflow tool node for them, because the code is a member. The agent path reads the field alone, so those refusals return to the model there.

the meter did not answer, or the caller passed an argument accrual cannot read
no accrual checker is wired
the call ran outside a run scope
a write or send tool reached no idempotency journal
policy asked for an approval and no approval service is wired
an approval needs a run scope, and the call carried none

The invoker reports four of the six to Sentry. Those four share one dedup set, so a fan out node reports each one time rather than one time per call.

denied, rejected, invalid_input and tool_unavailable set nothing, because each one tells the model to do something else. tool_changed will not either, when the connection resolver lands.

Three callers read it, and each already owns the answer.

Both agent rows call the one method. A second copy of the mapping would let the two paths end one run two ways, so AgnoToolAdapter.record_stop is public and holds it once. ToolStopFault carries the code the invoker named. It is terminal: _answer ends the run under that code rather than re-raising, because the same wiring answers the same way on every replay.

⚠️ The drain loop is the third caller, and it is the one a reader forgets. A write or send tool never reaches the adapter body. The framework pauses it, and the drain loop invokes it outside anything the framework wraps. A refusal delivered as that call's result lets the model propose the same send on the next turn, and the run then ends on max_agent_turns under limit_reached, which names the wrong clock. record_stop is public for that reason. Both paths call the one method, so one refusal ends one run one way.

The claim ends with the call

ok=True                          -> complete the claim with the stored result
ok=False, retryable=False        -> complete the claim; the answer is stable
ok=False, retryable=True         -> release the claim; nothing happened
the handler raised               -> release the claim; the segment retries
the call timed out               -> HOLD the claim; nothing is known

A retryable failure must not be remembered. A vendor 429 makes no effect, and a completed claim would return that 429 to every later attempt in the same run. The agent would then be locked out of a vendor that recovered a minute later. See idempotency.

A release is safe because the claim is taken before the handler runs. Nothing outside the platform saw the call.

⚠️ A timeout is the one ending that release does not fit, so it is its own row. The other four are knowable. A success and a stable failure are what the handler observed; a 429 is a rejection the handler read; a raise is a decision the handler made. A timeout is none of those: ToolInvoker cancelled the handler mid await, and the request it had already sent may still reach the vendor. Release it and the very next attempt sends the second email.

So a timed-out call completes nothing and releases nothing. The lease is what bounds it.

retry inside the lease   the claim reads `processing`, the invoker raises,
                         and Inngest replays the step after the lease
retry past the lease     the claim is reclaimed, exactly as it is for a
                         worker that died mid call

The second line is a real duplicate-effect window, and it is the same one a crashed worker already has. Only a downstream vendor idempotency key closes it, which is why passing the key downstream matters for a send.

The deadline case is the opposite answer, and it is also a TimeoutError. _execute refuses before the handler runs when the Run is already past its deadline. Nothing outside the platform saw that call, so it releases. The two are told apart by type, not by message: the pre-call refusal raises a subclass, and the invoker branches on it before the general timeout.

The approval is a pause, not a raise

A write or send tool is declared external_execution=True when the adapter builds it. Agno then stops its loop before that call runs, and RunOutput.status reads PAUSED. The declaration is static, and it says only "this call is worth a decision". The decision stays live: the runtime reads the pending call at the pause and invokes it with the arguments in hand.

model proposes email.send(to='jo@acme.test', ...)
  -> Agno pauses. The callable never runs.
  -> RunOutput.status = PAUSED, and each proposal is a RunRequirement
  -> the runtime reads requirement.tool_execution.tool_args
  -> ToolInvoker.invoke(...)          <- our code, at the pause, outside the loop
       allow            -> set_external_execution_result(<the ToolResult>)
       deny             -> set_external_execution_result(<a rejected ToolResult>)
       require_approval -> ApprovalRequired; the segment returns needs_approval
  -> acontinue_run(...), unless the segment stopped

The expiry still travels with the raise. AgentExecutionResult.approval_expires_at is what the Inngest wait computes its timeout from, and reading it back from the approval row afterwards would be a second query for a fact the raiser already held.

⚠️ The raise leaves the span block. It must not close the span error. The span recorder closes status = 'error' for every exception that crosses its context manager, and it publishes span.failed. An approval is not a failure. Every gated write would then read as a fault in the tree, beside the denials that close error on purpose, and an auditor could not tell the two apart. Catch the raise inside the block. Record the approval id as an attribute. Close the span ok. Then raise again, outside the block.

external_execution is what keeps the call in ToolInvoker. Agno's other pause, requires_confirmation, runs the callable itself once confirmed, which would move every write off the invoke path. With external_execution the platform executes the call and hands back a result, so ToolInvoker keeps the idempotency claim, the connection, the metering and the result boundary. Reserve requires_confirmation for a callable that genuinely belongs to Agno.

A read tool is not declared at all. It stays an ordinary Agno callable that the adapter runs inside the loop: there is no decision to make, so there is no reason to pay a pause for it. That split is what keeps the handler failure rules alive.

Measured on Agno 2.5.10 (ENG-2093), against ac-python-api/scripts/spikes/eng2093_agno_native_pause.py:

⚠️ requirements is not optional on the run_id path. Omit it and Agno looks for a resolved admin approval row and raises ValueError.

The runtime drains its own pauses

Every write and send tool pauses, so a call policy allows pauses too. That pause never reaches Inngest. execute() loops while the result is paused, resolving each requirement the invoker answers, and returns only when the loop finishes or a call needs a person.

result = await bridge.consume(agent.arun(
    input=segment_input.text,
    stream=True, stream_events=True, yield_run_output=True,
))
while result.is_paused:
    for req in result.active_requirements:
        stop.check_ceilings()              # before EVERY call, not once per pause
        try:
            outcome = await invoker.invoke(req.tool_execution, ...)
        except ApprovalRequired as exc:
            return AgentExecutionResult(stop='needs_approval', approval_id=exc.approval_id, ...)
        except CancelRequested:
            return AgentExecutionResult(stop='cancelled')
        if adapter.record_stop(outcome):        # a marked refusal ends the pass
            return _answer(result, stop)        # the siblings stay undecided
        req.set_external_execution_result(outcome.as_result())
    result = await bridge.consume(agent.acontinue_run(
        run_id=result.run_id, session_id=str(run_id), requirements=result.requirements,
        stream=True, stream_events=True, yield_run_output=True,
    ))

The continue streams like the first call, and bridge.consume is what makes both one shape. The bridge drives the event iterator, opens and closes the llm span, writes the usage row, feeds text_delta, and answers the final RunOutput. An unstreamed continue writes no heartbeat while a model turn runs, so the reaper fails a healthy segment at the first write tool. See the agent runtime.

The drain loop is the second place a ceiling is checked. A paused write call never reaches the adapter, so a run with nothing left would continue for ever on write tools alone.

⚠️ The check sits before every call, not at the top of the pause. Agno holds every proposal of the turn, so one pause can carry ten sends. Checked once, a remainder of one sends all ten and the ceiling stops only the next pass.

A second write proposed after a continue pauses again, and the loop drains that one too. Measured: two sequential sends, two pauses, two effects, and 3 provider requests against 3 unpaused. Agno marks a requirement it has handled, so a resolved one is skipped on every later pass.

One turn may hold several proposals, and each gets its own decision

Agno holds every proposal from the turn, not the first. So the decisions are independent, and a denial does not cost the calls beside it.

requirement 1  email.send(to='jo@acme.test')    -> allow  -> sent
requirement 2  email.send(to='sam@other.test')  -> deny   -> rejected result, model adapts

Measured: one effect, the allowed one. The model reported both outcomes truthfully.

A refusal the invoker marked is the one exception. It ends the pass at the call that met it, and the proposals beside it stay undecided. A spent ceiling ends the pass the same way, and for the same reason: the run is over, so a decision on the next proposal buys nothing.

A require_approval in a turn ends the segment. Calls already executed stay journalled, and the remaining proposals are simply not invoked. They return as requirements when the segment resumes, and the invoker decides them again with a fresh reading.

The resume executes the approved call

The runtime executes the approved call. It does not wait for the model to ask again.

This is the one place where relying on the model would be a correctness bug. The model is not deterministic. It may propose the same call, and it may not. A person who pressed Approve twenty hours ago would then see an approval marked approved and no email, with nothing in the tree to explain it.

The approval row already holds everything the call needs.

tool name · exact arguments · arguments hash · idempotency key · run · principal

So the resume path is deterministic, and it is a continue of the paused run rather than a fresh segment.

approval resolved = approved
  -> ApprovalService rechecks TTL, authority, arguments hash and target state
  -> ToolInvoker.invoke(..., authorized_by=approval_id)
       the action checkpoint is already decided; every other step still runs
  -> set_external_execution_result(...), then acontinue_run(...)

approval resolved = rejected
  -> set_external_execution_result(<a rejected ToolResult>), then acontinue_run(...)
  -> the loop continues, and the agent adapts

This is the drain loop with one extra argument, and not a second code path. The segment after approval rehydrates the paused run, meets the same unresolved requirements, and walks the same loop. authorized_by is passed for the one requirement the approval names, and every other requirement is decided live as before. Written as its own path it would carry its own copy of the ceiling check, the cancel rule and the requirement bookkeeping, and the two copies would drift.

A requirement the segment already executed before it stopped is met again on the resume, because the mutation never reached the stored session. The idempotency claim answers completed and returns the stored result, so the effect happens one time and the span carries replayed = true.

authorized_by answers the require_approval the person already decided. ⚠️ It skips no part of the decision. The engine still runs, so stage 1 still reads the grant live and a deny still refuses: a person offboarded while the run waited cannot execute the call they approved, and a rule an admin added while they deliberated still stops the effect. It skips nothing else either — the live tool state, the input schema, the journal read, the idempotency claim, the connection and the result boundary all still apply.

⚠️ A replay is answered before the checkpoint, and policy is not asked. The journal is read first, so a segment after approval that meets a call it already made returns the stored result. Policy is never asked to decide an effect that already happened. Only a write or a send journals, and the key names one organization, one run, one step and one arguments hash, so a replay is always one run reading back its own effect.

⚠️ The invoker checks that the approval covers the call, and the runtime cannot. One turn can hold several write proposals and a person answers one approval row, which names a tool and an arguments hash. The runtime holds neither, so it hands the same id to every proposal of that pause. An invoker that read the flag as pure trust would send a message a person never saw, on the strength of an approval for a different call.

An authorized_by that does not cover the call is ignored, and never denied. The flag skips one policy decision, so a flag that names another call skips nothing and the call is decided live. That call then raises ApprovalRequired again, and the run parks a second time on its own approval row. A denied here would drop that call permanently. The id names a different proposal of the same pause, and no later pass carries an id that names this one.

⚠️ A row that covers the call and reads rejected is the one exception. A person answered this exact proposal, so the invoker returns a rejected result before the engine is asked. Deciding it live would file a second proposal and show that person the same card again. "Covers" means the same four fields authorizes reads — organization, run, action, arguments hash — so a row naming another proposal of one pause is still ignored.

The claim carries that answer past the segment that received it. A later segment holds no authorized_by, and uq_approvals_run_id_idempotency_key releases its key on a terminal row since ENG-2156, so the insert would succeed and ask again. ToolInvoker therefore reads the rejection of <run_id>:<step_path>:<args_hash> before it files.

⚠️ That key is a claim, and never a proof. It names no tool, and step_path is one constant for an agent run, so two write tools whose validated arguments hash alike share one key. The invoker compares action and arguments_hash on the row it reads back. A guard that trusted the key alone would drop every later call of the second tool with no card shown, on the strength of a refusal of the first.

A read tool needs no such record. It takes no claim, and AgnoRuntime ends the Run on its first ApprovalRequired with read_tool_needs_approval: a read runs inside the agent loop, so there is no pause to carry a decision. A rejected read approval therefore cannot recur.

⚠️ A later segment is not bound by the rejection the way the handed segment is. The refusal above runs before the engine, so a rule an admin relaxes to allow does not release the call in the segment holding the id. In a later segment authorized_by is None, allow returns before the approval is filed, and the claim is never read. One call can end two ways in one Run, by the segment that holds the id. ENG-2156 took the person's answer over the rule on the path where both are known.

The second pause terminates. The segment after approval meets the call that already ran. The journal answers before the policy checkpoint, so that call returns its stored result and the pass reaches the next proposal. Each pass decides one more call, and no effect happens twice.

If the model does propose the same call again in that segment, the journal answers completed and returns the stored result. So the journal makes the deterministic path and the model path agree, and the effect happens once either way.

The paused call's result slot is empty, so nothing is edited. This is what the pause buys over a halted loop. A halt had already written a tool result for that call id carrying tool_call_error, so delivering the approved outcome meant editing Agno's message list from our code — exactly the knowledge the AgentRuntime boundary exists to contain. A paused call carries result=None, and the continue fills it. Measured: the invoker's result arrived on the call with error=None, on both the allowed and the refused path.

Handler failure

A handler under services/tools/handlers/ constructs no ToolError. It returns ok=True, or it raises. test_no_handler_constructs_a_tool_error holds that rule, because internal_error is a PLATFORM_STOP_CODES member and a handler that answered it would fail the whole run.

RetryableUpstreamError and UpstreamUnavailableError are neutral handler exceptions. They carry no vendor SDK type and no ToolResult. A vendor handler catches the provider class and raises one neutral class. ToolInvoker owns the stable error code and the retry flag. UpstreamUnavailableError maps to unavailable; tool_unavailable remains the code for a tool that is absent from the live registry.

Provider authentication, payment, request-shape and response-decode failures are not neutral upstream outcomes. A platform credential or a client contract is wrong, so the handler lets the error raise. A segment retry can then reach the normal retry limit and the span keeps the real fault. A decoded handler result that fails output_schema still follows the invalid_output row above.

The vendor retry budget fits inside timeout_s. The client timeout, every retry and every backoff must finish before the tool timeout. If the invoker cancels the handler first, it answers timeout and the handler never reaches RetryableUpstreamError. A declaration review checks the full budget, not one attempt.

An exception never crosses into the Agno loop except for a platform fault, where a segment retry is the correct answer and the journal bounds its cost.

⚠️ The last two rows are one decision, and an internal tool is where it is hard. A CRM read reaches Postgres, so most of what it can raise is the platform fault of row four. Turned into a value, a database outage becomes one failed call among several and the segment ends completed with a confident answer built on nothing — the failure the read path's catch-everything rule exists to stop.

So the invoker splits on the exception, and the list is explicit rather than a judgement:

a business exception the domain declares    -> ToolResult(ok=False) with a stable code
                                               NotFoundError, ValidationError, ConflictError
anything else                               -> raise; the segment ends and Inngest decides

⚠️ The test is the exact class, never the base class. ACError is the root of the AgencyCore error hierarchy and the wrapper every CRM service raises from its own except Exception — list_people answers ACError('Failed to list people') for a Postgres outage. So isinstance(error, ACError) reads a database fault as a business failure, which is the whole defect written as one convenient line. Match the leaf classes and let the base raise.

A handler that catches a database error and answers a value has moved a platform fault into row five by hand. Reviewing for that is the one thing a reader of a new handler must do.

Where the fault is raised decides how it leaves. A write or send tool pauses, so its ToolInvoker call happens in execute()'s drain loop, outside anything Agno wraps. Those raises are ordinary raises, and the runtime classifies each one. Nothing is swallowed, because nothing crosses the loop.

ApprovalRequired    -> stop = needs_approval, with the approval id and its expiry
CancelRequested     -> stop = cancelled
any other exception -> propagates; Inngest replays the step

CancelRequested needs its own row. A person pressed stop, and a raise that reached Inngest would land as a failure and overwrite that.

A read tool still runs inside the loop, and inside it the old measurement holds. Agno turns every exception raised in a tool into a tool result and continues. A database outage would become one failed call among several, and the segment would end completed with a confident answer built on nothing. So the adapter catches every exception a read tool's invoker raises, and it classifies rather than re-raises. It re-raises one, and the table says which.

a result carrying stop  -> record_stop() -> SegmentStop(ceiling|failed) -> StopAgentRun
CancelRequested         -> SegmentStop(cancelled)                       -> StopAgentRun
StopAgentRun            -> re-raised; the framework honours this one
any other exception     -> SegmentStop(failed, error=exc)               -> StopAgentRun

⚠️ The StopAgentRun row is load-bearing. StopAgentRun is an Exception, so a guard that caught every exception would catch the stop the adapter itself raised. It would then record that stop a second time and re-raise it under the fault message.

SegmentStop carries ceiling | cancelled | failed. needs_approval left it with the pause: an approval is decided where the loop is already stopped, so there is no signal to smuggle out. AgnoAgentRuntime re-raises the stored exception after the loop returns, so Inngest still sees a real failure and still decides the retry.

⚠️ Read the stored stop before RunOutput.status. StopAgentRun ends the loop and still answers a normal RunOutput, and a write tool proposed in the same turn makes that status read PAUSED. A runtime that checks the pause first drains a segment that already stopped.

Agno's own cancellation manager is not used. agno.run.cancel holds one manager in a module level global, so two segments of two organizations in one worker share it. The platform reads agent.run_control at the adapter instead, which is per segment and durable.

AgnoToolAdapter

The adapter holds no business logic.

read tool
  ToolSpec + ToolInvocation -> Agno callable
                            -> model proposes arguments
                            -> ToolInvoker.invoke(...) inside the loop

write or send tool
  ToolSpec + ToolInvocation -> Agno callable, external_execution=True
                            -> model proposes arguments
                            -> Agno pauses; the callable never runs
                            -> ToolInvoker.invoke(...) in the drain loop

The adapter declares the name a vendor accepts. The name bullet states the pattern. model_tool_name() replaces every character a vendor refuses with an underscore, so email.send reaches the model as email_send. It lives in models.py, beside NAME_PATTERN, because registration reads it too and services may not import runtime. The adapter holds the reverse map, which covers one segment because build() clears it.

The derivation is lossy, so registration owns the refusal. It replaces a character and never removes one, so two platform names can answer one model facing name. NAME_PATTERN allows one dot, so the pair reachable today is crm.search_people and crm_search.people, which both derive crm_search_people. The MCP phase widens the pattern and adds mcp.slack.send_message against mcp.slack_send.message. An mcp.<connection_slug>.<remote_name> can also exceed 64 characters.

build_registry therefore refuses a derived name that collides with a registered one, and a derived name over MAX_MODEL_TOOL_NAME_LENGTH characters. Truncation is not the answer, because it maps two names onto one again. The adapter refuses a collision again when it builds a segment, because it reads the frozen run snapshot and not the registry. An overwritten contract sends the right arguments to the wrong handler.

If the agent framework changes, the adapter is replaced. The registry, the invoker, the policy and the handlers do not move.

Result contract

class ToolError(BaseModel):
    code: str
    message: str
    retryable: bool = False

class ToolResultMeta(BaseModel):
    count: int | None = None
    truncated: bool = False
    untrusted: bool = False
    usage_id: UUID | None = None

# What the run does about a refusal the invoker made. Only the invoker writes
# it, and no caller renders it to a model.
ToolStop = Literal['budget_exhausted', 'fault']

class ToolResult(BaseModel):
    ok: bool
    data: Any | None = None
    error: ToolError | None = None
    meta: ToolResultMeta = ToolResultMeta()
    stop: ToolStop | None = None

Stable error codes:

denied                invalid_input          connection_required
rejected              tool_unavailable       connection_expired
not_found             tool_changed           connection_revoked
invalid_output        unavailable            connection_ambiguous
conflict              retryable_upstream     timeout
internal_error        batch_too_large
budget_exhausted

parent_cancelled is not in this list. A cancelled parent is refused by RunManager.start() before a child run exists, so it fails a workflow node and never reaches a model as a tool result. See runtime execution.

The model never sees a provider exception, a credential or an unbounded payload.

Output boundary

handler result
  -> validate the output schema
  -> remove platform credentials and tokens
  -> keep at most max_results items, when declared
  -> bound the size to max_output_bytes or the platform limit
  -> tag external content as untrusted
  -> set meta.count and meta.truncated
  -> span
  -> agent

Bounding is bound(output, 32 KB), and it never cuts a structure. The algorithm is defined once in the platform contract. This layer owns two things around it: meta.truncated comes from what bound() returns, meta.count stays honest, and redaction runs first, so a credential is removed by rule rather than by which item happened to be dropped.

The item bound runs before the byte bound. max_results applies only to an array output. It keeps the first items in order. max_output_bytes is at least 2, because [] and {} each need 2 bytes. It can lower the 32 KB platform limit and can never raise it. meta.truncated is true when either bound changes the payload. meta.count names the items that remain after both bounds.

An oversized scalar uses a Dropped marker when the marker fits. If a per-tool byte limit is too small for the marker, the boundary returns an empty JSON string and sets meta.truncated=true. The empty string is the smallest JSON scalar and fits the two-byte minimum.

output_schema validates the handler answer before the boundary. The bounded transport form can contain a Dropped marker or fewer items than the schema's minItems. The marker is one reserved platform envelope, and it is not copied into every node of every tool schema. A caller must use meta.truncated and handle the marker. Registration checks that max_results applies to an array, but it does not compare the limit with minItems.

The substitution reads the item it replaced, at the same position. bound() replaces every oversized item before it drops anything, and it replaces in place, so index i of the answer holds the marker for index i of the input, and a dict keeps its key. The invoker walks the two together and names the row from spec.item_kind and the item's own id.

The ref goes inside the marker, and does not replace it.

{"__dropped__": {"reason": "too_large", "bytes": 40960, "ref": {"kind": "crm.person", "id": "p-2"}}}

A bare ResourceRef sitting among real rows reads as a row: the model receives {"kind": …, "id": …} in a list of people and has no signal that an item is missing, or why. One sentinel key keeps the reason, the size and the pointer together, and __dropped__ is still the one lookup that finds it.

Two cases keep the marker with no ref. A spec that declares no item_kind names no kind, and an item that carries no id names no row. A ref built from a guess resolves to nothing, and a reader cannot tell that from a row a principal may not see.

⚠️ The naming can push the payload back over the limit, so the boundary bounds twice. A ref adds about sixty bytes per marker, and bound() ran before the naming. Measured on four hundred oversized rows: 42,691 bytes against a 32,768 limit, so the caller would store what it promised not to. The second call is the same function, not a second rule, and it drops nothing for a payload that already fits.

Redaction is a key name rule, not a value scan. A field whose key is a credential name — token, api_key, secret, password, authorization, and the two OAuth pair members — is replaced wherever it sits in the payload. Value matching would need a catalogue of every credential shape a vendor mints, and it removes a real body the first time a person writes about a password.

The span input is redacted too, and it is the arguments. The model wrote them. The span recorder bounds a span payload and relies on this layer having redacted it, so the invoker redacts the arguments before it opens the span, not after.

⚠️ One encoding measures the payload and renders it. Two encoders sat on this path and disagreed. The boundary measures with the database body encoder, which writes an Enum as its value, a datetime as ISO-8601 and compact separators. ToolResult.as_result() rendered with json.dumps defaults and default=str, which writes "Stage.ACTIVE", a space separated date, and two extra bytes per separator. The model read the second of each pair, and on 5,000 bounded rows it read 37,312 bytes against a 32,768 ceiling. So the boundary normalizes every value to plain JSON in its own walk, and the rendering uses the separators the measure used.

Every value the boundary answers is renderable, and the rule is total. The measure raises TypeError on a value it cannot encode, and that raise ends the segment — a set field and a bytes field each did it. Anything that is not a JSON scalar is therefore named rather than carried: "<unrenderable bytes>". A list of allowed types grows with every handler; this rule does not. A set becomes a sorted list, because the order of a string set moves with PYTHONHASHSEED and two workers would otherwise render two strings for one row.

Untrusted content is labelled, not scanned. V1 ships no prompt injection detector, and a detector we do not have must not appear in a design. The boundary does three things: it strips control characters, it bounds the size, and it sets meta.untrusted so the adapter wraps the body in a labelled envelope. The real defence is structural. External text can propose an action, and every action still needs a tool call that policy decides. A tool result never carries an instruction to the runtime.

Sensitive fields stay out of the output schema. V1 has no per tool redaction language. A field the model must not see is absent from output_schema.

Batch limits

max_batch_size is per call. batch_argument names the top-level list it measures. An oversized call is rejected after input validation and before the journal read, and it is never silently split. It answers batch_too_large. It reaches no policy read, claim, handler, preview, or meter.

ToolSpec.max_batch_size   what one call supports
Policy count and cost     what a run or an account may do
Vendor wrapper            vendor pacing and bounded 429 retry

Splitting is agent or workflow coordination. It is not tool infrastructure. A vendor rate limit is absorbed by a bounded retry inside timeout_s, and then returned as retryable_upstream. The platform adds no second limiter.

A vendor ceiling is arithmetic, not a limiter

max_batch_size bounds one call, and Inngest concurrency bounds the runs. Neither bounds one vendor across every live run. Two hundred runs each making a legal call can still cross an Exa or a FullEnrich account limit at the same second.

V1 does not answer this with a global semaphore. A semaphore is a second limiter, it needs Redis on the correctness path, and it fails in the one case it exists for, a worker dying while holding a slot.

V1 answers it with a size, chosen once and written down.

sum for each lane(
  active Run limit in that lane
  x maximum simultaneous calls to this vendor in one active Run in that lane
) <= the vendor account limit

Both terms are ours. Inngest sets the active Run limit per organization and lane. Within one Run, one wide node contributes its fanout_concurrency. A sequence and a branch take the largest reachable value. A parallel adds the values of its branches.

A child Run counts in its own lane. A suspended parent holds no slot, so the arithmetic never counts a child under its parent Run or its parent's lane. Add the interactive and batch lane products to get the account maximum.

This is a configuration review beside the vendor credential, not a run-time mechanism. Reading one node's concurrency would undercount two wide nodes in parallel. Validation still checks each node against the step budget. The bounded 429 retry handles short bursts that pass the account arithmetic.

Record the arithmetic beside the vendor credential, and check it when either number changes.

The escalation, when a vendor gets tight enough to need one: move that vendor call into its own Inngest function, keyed on the vendor name, and call it with step.invoke. Inngest then owns the ceiling, exactly as it owns run concurrency, and the platform still holds no limiter of its own. Do not build it before a vendor forces it.

Connections

ToolInvoker -> ConnectionResolver -> vault credential -> vendor or MCP server

Durable lifecycle:

connected -- auth or revocation failure --> needs_reauth
    ^                                         |
    +--------------- reconnect ---------------+

connected / needs_reauth -- uninstall --> revoked

connecting, refreshing, health checks and retries are operational states. They are not durable product states.

The model chooses the mailbox, never the credential. An organization may hold several connections of one provider, such as three Nylas grants. A tool that needs the choice declares a public connection_id argument. The resolver checks that the ID belongs to the organization and matches the provider, then injects the credential the model never sees.

Idempotency and the replay journal

Every write and every send reads the journal, then claims an idempotency key. The claim is never taken without the read before it, and both steps use one key, derived once. See the journal is read before the decision.

scope = tool.<tool_name>
key   = <run_id>:<step_path>:<args_hash>

step_path is the workflow node ID, or the literal agent. args_hash hashes the canonical arguments.

Do not key on the model tool call ID. An agent segment can restart after a crash, and the model mints a new tool call ID on every attempt. A key built from that ID never matches the earlier claim, so an approved send happens twice.

The claim record is the journal. agent.idempotency_keys already stores the status, the request hash and the response. A completed claim returns the stored result, which is exactly what a restarted segment needs. Do not add a second table beside it. See the note in what changes elsewhere.

Two identical write calls inside one run are one effect, on purpose. When a run must genuinely repeat one action, give the calls different arguments, or give the workflow two nodes, because the node ID separates them.

This rule has a sharp edge, so state it to a tool author. The journal cannot tell a replayed call from a deliberate repeat. Both look like the same arguments at the same step path. A replay must return the stored result, so a deliberate repeat receives it too, and the model reads a success for work that never happened a second time.

It is the right trade, because a duplicated send costs more than a skipped duplicate. It is only safe while tool authors know about it.

A tool whose repeat is meaningful cannot register in V1. repeatable = True is reserved for a later invoker that can carry a per-segment ordinal. This deployment refuses it rather than claim that it can separate a replay from a deliberate repeat.

read                 no journal key
repeatable = False   key = <run_id>:<step_path>:<args_hash>
repeatable = True    registration refused in V1

Registry validation cannot decide whether identical calls are one effect, so every write and send tool must declare repeatable = False. A read declares no value because it takes no claim. A tool such as crm.append_note stays unavailable until the runtime has a crash-safe repeat rule.

When a vendor supports idempotency, pass the same key downstream. That closes the crash after vendor success window.

Cost and spans

Every tool call opens one span. A metered call also writes one usage row through the canonical meter, and the span links it with usage_id.

This is not decoration. Policy accrual reads that meter, so a Signals Search run that spends money at Exa and FullEnrich must stop at its ceiling like a run that spends money on tokens. A vendor call that is metered nowhere is a hole in the budget.

Only a successful billable response writes a usage row. A policy or accrual refusal runs before the handler and writes none. An exhausted retryable failure also writes none. The vendor did not complete a billable operation that the handler can price.

An asynchronous provider job is the one exception, and it is not a second rule. The billable operation still writes exactly one row, and the row is still written from what the provider reported. Only the frame moves: the provider answers through a callback, so a durable function writes the row instead of the handler. See asynchronous provider jobs.

The handler calls a legacy vendor client with no usage logger. It then writes one UsageRecord through UsageMeter.record(). The row carries the ambient root_run_id, and the handler puts the returned row id in ToolResult.meta.usage_id. A meter failure raises. The handler never returns vendor data that the run cannot count.

The metered checkpoint already proves that the call has an ambient root run id before the handler starts. The handler reads the same id for the row. This order prevents a paid call that cannot name its run tree.

When a provider returns usage, use it. Firecrawl returns the credit count for a scrape, so the row multiplies the per-credit price by that count. A fixed one-credit row can undercount an enhanced proxy or a multi-page document.

Research tools

These tools are reads. They take no idempotency claim. research.search_company uses the low-latency Parallel Search endpoint that the current async client exposes. Parallel Task is the structured enrichment API, but it can run for hours. It is not an agent-loop tool. It is a submit tool and a collect tool over asynchronous provider jobs.

An empty Exa result is a successful billable response. It returns an empty list and writes one usage row. An exhausted 429, transport fault or provider timeout raises RetryableUpstreamError and writes no row. The strict research path must keep these outcomes distinct even though legacy Exa callers can still fail open.

The research.discover_people tool takes one provider because one tool span links to one usage row. The workflow fans out company and provider pairs when it needs wider coverage. One call never hides two vendor charges behind one usage_id.

The buyer persona can name a title, title family, seniority, function and location. The handler builds the vendor query from those bounded fields. It returns a common candidate projection and never returns the vendor payload.

research.find_contact answers one address the provider returned, and never one this platform built. A miss answers not_found, keeps the person and prices the usage row at zero. Hunter charges one credit for an answered address alone.

The answer carries one closed verification state: verified, unverifiable, accepts_all, unverified or not_found. The provider status outranks accept_all, because a domain that takes every address still tells the truth when it names one address invalid. A role mailbox such as info@ is flagged and kept, so the agent decides.

Two client behaviours make that rule necessary.

HUNTERIO_DRY_RUN makes the client answer synthetic data and reach no vendor. It builds first.last@domain for the email finder, and it builds the company name from the domain for the enrichment. The deploy passes dry_run=False, and both Hunter handlers also refuse a dry run themselves. One guard reads the client before the call, so a dry run spends nothing. One guard reads the _dry_run marker on the row before the meter, for a client that carries no attribute. The rule is absolute, so it holds at the handler and not at one call site.

HunterIOClient.find_email folds a 400, a 401, an unreadable body and an unexpected fault into the None a real miss returns. research.find_contact therefore passes strict=True, and a bad request raises instead of reading as a person with no email.

crm.search_people is a different tool. It reads people that already exist in the organization's CRM. It does not call a research vendor.

Every research declaration sets kind='vendor' and metered=True. Each handler sets meta.untrusted=True. The list tools set max_results=10. The common byte boundary applies after output validation.

Asynchronous provider jobs

Some providers finish work after the call that started it returns. FullEnrich is the first. Registry validation refuses a timeout_s over STEP_BUDGET_S, which is 120 seconds, so a job that runs for an hour cannot be one tool call. The 300 second bound on the field is the outer limit of the type and no tool reaches it.

The platform answers this with four parts and one new table. A tool submits, a webhook stores the outcome, a durable function settles the cost, and a second tool reads the result.

tool  fullenrich.submit  claim -> reserve job -> one vendor call -> bind vendor id -> complete claim
      (the run parks on the job, see the wait below)
POST  /api/v1/agentic/webhooks/{provider}/{job_id}
                         verify -> read the reserved job -> bind vendor id -> store the transition
                         -> claim webhook.<provider> -> enqueue -> ACK
fn    provider_job.settle  write one usage row, once
tool  fullenrich.collect read one tenant job, return running | completed | failed | cancelled

The record

agent.provider_jobs
  id                   the platform job id; the correlation key
  organization_id
  root_run_id          the tree the cost belongs to
  run_id               the run that submitted
  tool_call_id         the span of the submit call
  provider             'fullenrich'
  provider_job_id      the vendor's own id; null before confirmation
  contact_refs         the ordered, unique refs in the submit request
  state                submitted | running | completed | failed | cancelled
  result jsonb         at most 32 KiB; larger results use provider_job.result refs
  cost jsonb           what the provider reported
  usage_id             the one usage row; null until settlement
  submitted_at, updated_at, terminal_at

unique(provider, provider_job_id)

The correlation key is id and never provider_job_id. The platform id is ours, it is stable, and it is minted before the vendor answers. A workflow correlates on a value it can read from the submit step.

FullEnrich submit and collect

fullenrich.submit is a metered write. fullenrich.collect is an unmetered read. Both names require explicit rights. Neither calls enrich_and_wait or polls. The submit journal stores the small answer, including job_id; it stores no separate resource pair.

Submit accepts 1 to 25 contacts. Each contact needs a unique ref of 1 to 64 characters, first and last names of 1 to 120 characters, and a company domain, company name or LinkedIn URL. Company names have at most 200 characters. Domains are normalized. LinkedIn URLs have at most 512 characters. The tool accepts no callback URL, organization, run ID, credential or arbitrary custom fields. The provider receives work-email fields and custom.ref only. The deployment supplies the batch callback URL.

The platform derives job_id from organization, run, step path and the validated arguments hash. It inserts the job with ordered contact refs before calling the provider. The insert winner alone sends one POST. Replays and concurrent losers return the same reserved job. A null provider_job_id means acceptance is unknown, not that the provider refused the job. The submit response or a verified callback can bind this ID once. Neither can change an existing ID. The batch name is agentic:<job_id>. Before callback binding, require this exact name in the signed body. The signature alone does not bind the URL path. A callback for another reserved job must fail this name check.

There is no automatic POST retry. FullEnrich documentation does not establish support for Idempotency-Key. A timeout, lost response or worker failure therefore leaves the reservation for callback recovery. A replay must not submit it again, even after the tool journal expires. A worker failure before the POST can leave an unsubmitted reservation. The workflow wait then times out. This contract prevents duplicate batches; it does not promise a successful submission after every process failure. A definite HTTP rejection raises the provider error and leaves the same reservation. It does not infer terminal provider cost.

Submit returns job_id. Collect accepts only job_id and filters by organization and provider in the database. An unknown job and another tenant's job both return not_found. Collect returns the job ID, mapped state, acceptance confirmation, and one compact result per requested ref. A live job returns no contact results. A terminal job preserves partial success, including failed or cancelled batches. A missing email does not make a completed batch fail.

Collect maps only exact echoed refs. It never uses position, name or domain as a substitute. Duplicate known refs produce no email for that ref. Missing refs and malformed rows produce explicit diagnostics. Unknown refs are counted and ignored. Diagnostics never copy arbitrary provider text. Each result carries the ref, email, provider status, confidence and a result status. Email and provider status have fixed length limits. Verification is derived only from an explicit provider grade. The regular tool boundary applies after projection. The full 25-contact projection must fit its 32 KiB limit without loss.

Large provider results

agent.provider_job_results owns the complete provider result as private data. Each row has the job ID, result digest, organization and result. A composite foreign key prevents a tenant mismatch. Rows cascade with their job. RLS is enabled; only the service role has table access. The key is (job_id, sha256(canonical result)). Insert-on-conflict-do-nothing preserves each immutable body. The conditional job transition selects one state, cost and result reference together. A competing callback cannot pair its cost with another callback's body. Events use the stored terminal state.

The callback body has a 2 MiB limit. The shared bound() helper checks the 32 KiB job payload limit. A result that fits stays inline. A larger result is stored before the terminal transition. The job then stores {"kind": "provider_job.result", "id": "<job_id>/<result_digest>"}. A failed result write prevents the transition and ACK, so a callback retry can recover. Collect resolves only this kind, the same job ID and that digest, with an organization filter. A missing referenced result is a storage fault, never an empty success. Existing inline jobs remain readable. A job with no stored contact refs cannot invent a mapping.

The callback URL carries the job id

A vendor callback carries the vendor's job id and no organization. The claim is organization scoped, so the tenant must be known first.

The platform mints id before it calls the vendor, so it builds the callback URL from it. FullEnrich takes a webhook_url for each batch. The route is therefore per job, and the tenant comes from the path:

POST /api/v1/agentic/webhooks/{provider}/{job_id}

1. resolve the adapter from the path          unknown provider -> 404
2. rate limit by client IP
3. verify the signature over the raw body     bad signature -> 401
4. read agent.provider_jobs by id             the tenant, from the reserved row
5. unknown job id  -> drop reason `unknown_job`, then ACK
6. bind an absent vendor id, store a large result, then conditionally transition
7. a progress update ends here -> drop reason `not_terminal`, then ACK
8. IdempotencyService.claim(org, 'webhook.<provider>', delivery_id, sha256(body))
9. durable enqueue                            -> Inngest
10. ACK                                       -> under 3 seconds

Step 6 runs before step 9. The durable truth is the row, and the event is a wake-up signal alone.

⚠️ Steps 8 and 9 run for a terminal update alone. A progress callback records what it says and stops. It takes no claim, so it cannot make the terminal delivery a hash conflict. The rule and its reason are under completion emits one versioned event.

⚠️ Nothing unverified reaches the database. The adapter comes from the path and the signing key is one platform-wide secret per provider, so verification needs no tenant. Reading the job first gives an unauthenticated caller one indexed read for each request, and it answers a different status for a job id that exists. That difference is an oracle over the id space.

The job id in the path is not a secret, and nothing here treats it as one. Step 3 gates step 4, so an attacker who learns a job id still changes no state without the signing key. The path answers which tenant, and the signature answers whether to believe the body.

FullEnrich posts once, on the terminal state. ENG-2300 captured one live batch. The vendor sent no callback at CREATED and none at IN_PROGRESS. The body carries id, name, status, cost and data, the status is uppercase, and cost.credits counts hits and not rows. It signs with X-Signature-SHA1 over the raw body and carries no delivery id header, no timestamp and no retry counter. FullEnrich also exposes a second callback URL for each contact (webhook_events.contact_finished), and that URL does report progress. No tool sets it.

A provider that takes no per-request callback URL reads the job by (provider, provider_job_id) instead. That pair is unique across every organization, so it resolves one tenant. It is the fallback and not the shape to copy, because it needs one cross-tenant read.

⚠️ Every provider signs with one platform-wide key in V1. verify() takes the raw body and the headers, and nothing else, so it cannot check a secret held per tenant. A provider that needs one is a change to this Protocol and not a configuration choice, and no provider needs one today.

⚠️ An unknown job id is not an error. A retry for a job this platform already deleted reaches no operator, and a 5xx would tell the vendor that a delivery it made successfully failed. The handler counts unknown_job and answers its ACK, exactly as a duplicate delivery does.

Duplicate and out-of-order callbacks settle to one transition. Step 6 is conditional on the legal from-states, so a terminal job stays terminal. This is the rule runtime execution applies to the Run row, applied to a second row.

Completion emits one versioned event

event type    agentic.provider_job.completed.v1
Inngest name  platform/agentic.provider_job.completed.v1
event id      event:<provider delivery id>
data.data     job_id     the platform id, and the correlation key
              provider, state, terminal

⚠️ The correlation field is job_id and never provider_job_id. The row already has a column of the second name, and it holds the vendor's id. One name for two values is the defect that reaches production as a wait that matches nothing. The submit tool returns job_id, the event carries job_id, and a match reads job_id.

⚠️ A terminal update emits, and a progress update does not. A progress callback stores its transition and ends there. It takes no claim either. A wait returns on the first matching event and reads no row again, so a running event wakes a run on work the provider has not finished. The claim compounds it: the key is the delivery id, so a progress delivery that claimed it makes the terminal delivery a hash conflict, and the run then parks for its whole timeout with the result already stored. ENG-2300 measured this.

The version is in the event type. See triggers for the rule that governs it.

The Inngest event id gives short-term deduplication over a 24-hour window. It is not the durable guard. The webhook.<provider> claim and the conditional transition are.

The run parks on the job, and two reads close the gap

A wait node whose correlation names a provider job reads that row once before it registers the wait, and once again on the timeout. Those two reads are the mechanism the approval wait already uses. See the wait node.

⚠️ Without the first read the run parks for the whole timeout. The submit step commits, the callback lands, and the wait registers after the event is gone. The job row survives that gap, and the read finds it terminal.

ENG-2300 measured one live FullEnrich batch. The submit call answered in 0.83 seconds and the one callback arrived 307 seconds later, so the gap is wide for a batch that does real work. A batch that ends at once closes it: a drained account reports its ending immediately, so the submit tool reserves the row before it sends the request.

A parallel node holds at most one parking subtree, so it holds at most one provider-job wait. RunManager.resume() counts pending approval rows and does not count provider jobs. Lifting that rule is not V1 work.

The cost settles once, off the run frame

provider_job.settle is an Inngest function on the completion event. It writes one UsageRecord through UsageMeter.record(), stamped with the job's root_run_id, and then stamps the row:

UPDATE agent.provider_jobs SET usage_id = :usage_id
 WHERE id = :job_id AND usage_id IS NULL

The meter uses a deterministic usage ID derived from the platform job ID. Its existing upsert prevents a second charge if the process fails before step acknowledgement. Zero rows from the stamp means another attempt settled it. Cancellation, timeout, a replayed callback and a repeated collect all read the same stamped row and change no charge.

Settlement is not on a run frame, and this is deliberate. A cancelled run and a timed-out wait both end before the vendor answers. The money is still spent, so the row still names agent_root_run_id and the tree total grows after the Run is terminal. A total that hides it understates a tenant's bill.

⚠️ This is the one metered path that does not end a Run on a meter failure.Observability states that UsageMeter.record() raises and the caller ends the Run. There is no Run to end here, and the Run that submitted is often already terminal. So the function raises, Inngest retries it, and an exhausted retry reports to Sentry and leaves usage_id null. A null usage_id on a terminal job is the query that finds an unsettled charge.

provider.collect writes no usage row. It reads the job row. A read takes no claim, so repeating it is free.

row state              collect answers
submitted, running     running
completed              completed
failed                 failed
cancelled              cancelled

submitted and running collapse, because a caller does nothing differently between them: the work is not finished either way. cancelled does not collapse into failed. The provider stopped the job, and a workflow that retries a failure must not retry a job somebody cancelled.

FullEnrich design stress cases

ENG-2210 owns these cases.

The provider adapter

One Protocol and one implementation per provider. The deployment maps provider names to adapters.

class ProviderWebhookAdapter(Protocol):
    provider: str
    def verify(self, raw_body: bytes, headers: Mapping[str, str]) -> bool: ...
    def delivery_id(self, payload: dict) -> str: ...
    def read(self, payload: dict) -> ProviderJobUpdate: ...

ProviderJobUpdate carries the vendor job id, the next state and the reported cost. The vendor job id is the vendor's own, and the route compares it against the row the path named. The adapter verifies authenticity and maps the vendor's vocabulary. It touches no Run, no meter and no span.

verify() runs before delivery_id() and read(), and the route holds that order. The path names the job, so no method reads an unverified body to find the tenant. read() answers trusted fields alone, and it runs after the signature passes.

The vendor job id read() returns must equal the one on the row. A signed delivery for job A that names job B in its body is a provider defect or a replay across jobs. The handler counts job_mismatch and answers its ACK.

The platform refuses a registry of adapters for the same reason it refuses a general plugin framework. One dict keyed on the provider name resolves the route parameter, and a route parameter that names no adapter answers 404.

provider_job.settle retries three times, which is the count every metered path uses. An exhausted retry reports to Sentry one time and leaves usage_id null.

Retention, and what a vendor still owes

agent.provider_jobs is deleted with its Run. The composite run key cascades, so the 13 month agent.runs window is this table's window and retention.sweeper needs no fourth sweep. A job outlives the 90 day span window on purpose: the span that submitted it is gone long before the row is.

Pick timeout_s from the provider's own stated completion time, not from the 30 day ceiling. A wait that outlives the Run's max_run_duration_s is clamped to the Run deadline, so a generous timeout does not park a Run for ever. A timeout under the provider's real latency does: the wait ends, the Run moves on, and the callback settles a cost for work the Run never read.

A vendor with no callback needs a poll instead. V1 builds no poll backstop: the wait timeout ends the Run, and a job whose callback never arrives keeps state at its last transition. A provider that drops deliveries needs a reconcile sweep before it ships, and it is not built before a provider forces it.

Cancellation

A tool call is a safe boundary, so the executor reads agent.run_control before it and after it. The invoker never interrupts a call in flight. If the effect is already sent, let it settle, record it, then stop.

Registry validation

build_registry runs when the process builds its object graph, and the MCP resync will run the same checks when a connection syncs. It does not run per call.

⚠️ It does not fail a deploy. The graph builds on the first function run and on the first request, so a broken declaration fails those two. Every check below therefore raises RegistryError, whose message a person reads.

It checks:

• the name is unique, valid and not a platform collision;
• the derived model facing name is unique, and at most 64 characters;
• the input and output schemas are valid JSON Schema;
• the handler or the connection binding resolves;
• approval_preview=True resolves to a handler with a callable preview();
• side_effects is known, and a send tool declares a connection;
• every write and send tool declares repeatable = False, a read leaves it unset, and True is refused;
• every count limit is positive, and max_output_bytes is at least 2;
• batch_argument and max_batch_size appear together, and the argument names a top-level array;
• max_results appears only on an array output, and max_output_bytes does not exceed the platform limit;
• every name in policy_facts has a registered resolver, and none of them is principal, run or arguments;
• timeout_s is inside the step budget.

A broken tool never enters the runnable registry.

Scenarios that shaped this design

Rules

• An agent reaches the world only through a tool.
• An agent never touches Postgres, a raw route or a credential.
• An agent sees only the tools its definition names.
• Visibility, policy, business validation and tenancy stay four checks.
• Visibility is the frozen snapshot. Authority is stage 1 of the policy engine. There is no factory between them, and the invoker holds no scope check of its own.
• Policy is outside the agent, and it is evaluated on every call.
• A write or send tool is declared external_execution. It pauses, and ToolInvoker runs it at the pause.
• The runtime resolves in process every call the invoker answers. Only an approval reaches Inngest.
• One turn may hold several proposals, and each gets its own decision. A denial returns as a value.
• A read tool runs inside the loop, so the adapter catches every exception it raises and classifies it. It re-raises StopAgentRun, which is the stop it raised itself.
• SegmentStop carries ceiling | cancelled | failed, it belongs to the segment, and the first writer wins.
• ToolResult.stop is written by the invoker alone. _bound drops a value a handler wrote, and no caller renders it to a model.
• A metered call refused at the run cost ceiling ends the run partial in all three callers. A fault ends the run failed under the result's own code.
• The drain loop is the third caller. A write or send tool never reaches the adapter body.
• The approved outcome fills the paused call's own empty result slot. No message is edited.
• The resume executes the approved call. It never waits for the model to propose it again.
• An authorized_by the approval row does not cover is ignored. The call is decided live, never denied.
• A covering row that reads rejected answers rejected, and so does a claim a person already refused. Neither files a second proposal.
• ApprovalRequired closes the tool span ok. A pause is not a failure.
• Every write and send claims an idempotency key, and the claim is the journal.
• The journal is read before the policy checkpoint, and the claim is taken after it.
• A live processing claim on the call's own key raises. It is never a ToolResult.
• ToolInvoker opens the tool span before the checkpoints, so a refused call is in the tree.
• Span identity is ambient. No invocation, request or handler carries a copy of it.
• An internal handler reaches organization data through scoped_db, never through the admin client.
• A global Intelligence handler calls dedicated Intelligence domain stores. Only those stores use the admin client.
• An internal handler answers a value for a business failure, and raises for everything else.
• A vendor handler raises a neutral upstream exception. The invoker owns the stable error and retry flag.
• A claim ends in complete or release. A retryable failure releases it.
• A metered tool checks accrual before it executes.
• A vendor ceiling is arithmetic over the run concurrency, not a limiter.
• A tool whose repeat inside one run is meaningful carries a distinguishing argument.
• Every call opens one span, and a metered call writes one usage row.
• A successful billable call writes one usage row. A refusal or retryable upstream failure writes none.
• The boundary bounds the result without breaking its structure.
• External content is labelled untrusted, and it is never treated as an instruction.
• Connections hold three durable states, and the model never sees a credential.
• Every stored name keeps its dot. model_tool_name() derives the model facing name, and the adapter alone applies it to a segment.
• Registry validation catches declaration drift before run time.
• preview() is read only, runs before authorization, and may fail without losing the approval.
• Output that fails its own schema is invalid_output, never internal_error.
• A remote schema change bumps the spec version, and a run on the old contract stops with tool_changed.
• Every write and send tool declares repeatable. Silence is not an answer.
• A metered call reads the run tree total. The organization day belongs to the node boundary.
• An asynchronous provider job is one row, and that row is the durable truth. The completion event is a wake-up signal, never the record.
• A provider callback resolves the tenant from the job row before it takes an organization scoped claim.
• A provider callback stores its state transition before the platform emits the completion event.
• A provider job state transition is conditional on the legal from-states. A terminal job stays terminal.
• The provider cost settles one time, and a conditional write on a null usage_id is the guard.
• A collect tool reads the job row. It calls no provider, writes no usage row and starts no second job.
• A wiring fault on the metered path reaches Sentry one time. The log receives every call.
• Accrual reads ai_usage_log, never the daily rollup.
• Organization authored executable tool code stays deferred. Use platform tools, approved MCP servers and organization skills first.

Open decisions

1. Does a send tool need a per organization daily ceiling of its own, or is the policy count limit enough?
2. Which tools set approval_preview in the first release, beyond email.send?
3. Which vendor forces the per vendor Inngest function first, and at what account limit? FullEnrich forces the asynchronous job contract, which is a different mechanism: it moves the completion off the call, not the rate limit off the run.
4. Is an hourly MCP resync the right exposure window, or does a busy server need a webhook?

Minimum contract tests

• A replayed submit returns the same provider job and makes no second vendor call.
• A callback that lands before the wait registers still ends the wait, and the run does not park.
• Two deliveries of one callback produce one state transition and one usage row.
• A callback that reports an earlier state than the stored one changes nothing.
• A callback for an unknown provider job id counts a drop reason and answers an ACK.
• A callback whose signature fails changes no state.
• The cost settles once when the run was cancelled before the callback arrived.
• Repeated collection answers the same result and writes no second usage row.
• The Agno adapter cannot bypass a policy denial.
• A database fault inside a tool ends the segment, and the model claims nothing.
• Two segments in one process do not read each other's stop signal.
• A call policy allows pauses, is granted in process, and never reaches Inngest.
• A write tool's callable is never executed by Agno; every effect goes through the invoker.
• Two proposals in one turn get two decisions, and a denial does not stop the call beside it.
• A required approval ends the segment, and the resume executes the tool once into the paused call's slot.
• A required approval closes its tool span ok, carrying the approval id.
• A resumed pause holding a second write call decides it live, and parks the run on its own approval.
• An oversized batch produces zero side effects.
• An oversized batch reaches no journal read, policy read, claim, handler, preview or meter.
• Batch argument and batch size must appear together, and the argument must name a top-level array.
• Result count and byte limits compose, and either limit sets meta.truncated.
• A one-byte output limit does not build.
• A preview is one non-empty line of at most 512 UTF-8 bytes.
• A preview timeout or invalid result files the approval with preview=None.
• A crash after a vendor effect does not repeat the effect.
• A turn holding two gated write calls finishes in three passes, with two effects.
• read() writes no row, so a call that stops for a person leaves no claim behind.
• A crashed worker's claim is read as absent, and the next attempt reclaims and executes it.
• A live claim on the call's own key raises, and the model reads no result for it.
• A restarted segment returns completed journal results. Interrupted and new calls follow the claim, lease, and vendor recovery rules.
• An expired, revoked or ambiguous connection maps to a stable error code.
• A revoked tool is refused after an approval resume.
• A handler result is valid against output_schema before the boundary changes its transport form.
• One item larger than the whole limit keeps its shape, with its big fields replaced by a ResourceRef.
• Redaction runs before bounding, so a credential in a dropped item is still removed.
• An external secret never appears in a ToolResult or a span payload.
• A metered vendor call reaches the usage meter and the accrual checkpoint.
• A metered call is refused when the run is already over its cost ceiling.
• A metered call with no root run id reaches no handler and writes no usage row.
• One successful vendor response writes one row, and its id reaches the tool span.
• A retryable upstream failure writes no usage row.
• A vendor handler constructs no ToolError; the invoker maps its neutral exception.
• A valid empty Exa response is distinct from an exhausted retryable failure.
• An unsupported Firecrawl target maps to unavailable and writes no usage row.
• A provider credential fault or a meter fault raises and returns no vendor data.
• Vendor arithmetic adds two wide nodes that can run in parallel against one account.
• The full vendor retry budget fits inside the declared tool timeout.
• A Firecrawl row uses the response credit count when it is available.
• Many metered calls that meet the same wiring fault produce one Sentry report, and one log line each.
• A retryable failure leaves no claim, and the next identical call executes.
• A business failure leaves a completed claim, and the next identical call returns it.
• An approved call executes on resume even when the model proposes nothing.
• A rejected approval returns a rejected result into the loop, and the run continues.
• Schema and handler drift fails validation before publication.
• A handler returning off schema data produces invalid_output.
• A failing preview() still creates the approval, with the generic render.
• A preview performs no write, no send and no metered call.
• A remote input schema change makes the next call return tool_changed, not invalid_input.
• A tool that declares repeatable = True cannot register in V1.
• A non repeatable tool called twice with identical arguments writes one.
• A send tool cannot register with repeatable = True.
• A run principal carries the tool names its definition names, and the invoker refuses a name it does not hold.
• A Dropped marker becomes a ResourceRef only when the spec names an item_kind and the item carries an id.
• A domain business exception is a value, and every other exception raises and ends the segment.
• A spec whose remote schema changed carries a new schema_hash and a higher version.
• A dotted platform name reaches the model derived, and a paused call maps back to the dotted name.
• Two contracts that derive one model facing name are refused before the model sees either.
• A derived name over 64 characters is refused at registration.
• A tool result never carries parent_cancelled.