State and knowledge

The runtime decides how to run work. This layer decides what the work may know.

Reading is this layer. Acting is a tool.

Fact owners

conversation > application state > definition summaries > knowledge > memory > run history

Six classes, because ContextItem.kind has six values. Definition summaries sit between application state and knowledge: they explain published Agents, Workflows and Skills, but they do not override current application rows. Run history sits last: it is the weakest claim about what is true now, since it says only what an earlier run believed. A shorter order would leave the packer with no rule for the rest.

Conversation sits first, and it is the one class that is never cut. It is what the person said, so a brief that dropped it would answer a question nobody asked. It creates no subject_key, so it never wins or loses a dedupe.

What precedence actually does

Precedence is a packer rule, not a sentence in a prompt. It does three things, and nothing else.

1. Section order. The brief renders the conversation first, then application state, then definition summaries, then knowledge, then memory.
2. Budget order. The packer spends the token budget from the top. Lower-priority classes are cut before higher-priority classes.
3. Same subject wins upward. When two items share a subject_key, the higher class stays and the lower class is dropped, and the drop is counted.

The key format, because precedence is nothing without it

Rule 3 fires only when two items create the same string. The source contract defines each attribute namespace, so a key collision is deliberate.

subject_key = <entity kind>:<entity id>:<attribute>

crm.company:8f21…:lifecycle_stage
crm.person:4a09…:current_title
user:3b7c…:preference.language

A key is optional, and an item without one never collides. That is correct: most knowledge chunks are about a subject too broad to name.

One make_subject_key() helper creates every key from a ResourceRef and an attribute. It returns no key when the ref or attribute is None.

It raises ValueError when a part is empty or contains :. These rules make the delimiter unambiguous.

The attribute half is what makes it work. crm.company:8f21… alone would merge unrelated facts about one company. Two items collide only when the full key matches.

Application attributes are exact field names. Memory attributes stay in the observation. and preference. namespaces. These keys do not collide by accident.

The packer cannot detect that a memory sentence contradicts a CRM row in free text. It can enforce these three rules. Do not claim more.

Context builder

The context builder is deterministic infrastructure. It is not an agent, and it makes no model call.

ContextRequest + ContextPolicy
             │
             ▼
       ContextBuilder
             │
      concurrent retrieve
     ┌───────┬────────┼────────┬────────┬──────────┐
     ▼       ▼        ▼        ▼        ▼          ▼
Conversation Application Definitions Knowledge Memory Run history
     └───────┴────────┼────────┴────────┴──────────┘
             ▼
        ContextItem[]
             │
             ▼
        ContextPacker
             │
             ▼
        ContextBrief

Core code

services/
  crm.py          the CRM projections, the ref normalizer and the scoped read
services/context/
  models.py       ContextRequest, ContextPolicy, ContextSourceSpec, ContextItem, ContextBrief
  builder.py      ContextBuilder
  packer.py       ContextPacker
  registry.py     ContextSourceRegistry
  sources/
    application.py    CRM and product rows already in scope
    conversation.py   the recent messages and summary of one conversation
    definition_summary.py  safe summaries of visible Agents, Workflows and Skills
    knowledge.py      hybrid search over knowledge chunks
    memory.py         agent.live_memories
    run_history.py    earlier runs for this subject or entity
services/memory/
  models.py       MemoryCandidate, AgentMemory
  repository.py   service-role memory reads and inserts
  service.py      MemoryService

Contracts

@dataclass(frozen=True)
class ContextSubject:
    organization_id: UUID             # the tenancy every source filters by
    user_id: UUID | None
    trigger_id: UUID | None
    definition_id: UUID | None        # None for a front door turn


@dataclass(frozen=True)
class ContextRequest:
    subject: ContextSubject           # every source filters through this
    query: str                        # user text, or the run input summary
    entities: list[ResourceRef]       # what is already in scope
    run_id: UUID | None = None
    conversation_id: UUID | None = None


class ContextSourceSpec(BaseModel):
    name: str                         # the registered source name
    enabled: bool = True
    required: bool = False            # a failure here fails the run
    token_budget: int = Field(ge=0, strict=True)
    limit: int | None = Field(default=None, ge=0, le=MAX_CONTEXT_ITEM_LIMIT, strict=True)  # None uses the source default
    options: dict = {}                # validated against the source's declared schema


class ContextPolicy(BaseModel):
    sources: list[ContextSourceSpec]
    total_token_budget: int = Field(default=0, ge=0, strict=True)
    refresh_on_resume: bool = True


class ContextItem(BaseModel):
    source: str                       # which source produced it
    kind: Literal['application', 'conversation', 'definition_summary', 'knowledge', 'memory', 'run_history']
    title: str
    body: str
    subject_key: str | None = None    # precedence and dedupe
    ref: ResourceRef | None = None    # the row behind the item
    recorded_at: datetime | None = None
    tokens: int = 0


def make_subject_key(ref: ResourceRef | None, attribute: str | None) -> str | None: ...


class ContextBrief(BaseModel):
    text: str
    items: list[ResourceRef]          # what the model was shown, for audit and citation
    tokens: int
    per_source_tokens: dict[str, int]
    dropped: dict[str, int]
    failed_sources: list[str]


class ContextBuilder:
    async def build(self, request: ContextRequest, policy: ContextPolicy) -> ContextBrief: ...


class DefaultContextBuilder:
    def __init__(
        self,
        registry: ContextSourceRegistry,
        packer: ContextPacker,
        *,
        source_timeout_s: float = 1.0,
    ) -> None: ...

⚠️ The request carries a subject, and it does not carry a Principal. Principal is the frozen authority of one run, so run_id and definition_id are required on it. The Front Door builds context before any run exists, and it must never mint a principal, because a fake authority is what the policy plane exists to refuse.

ContextSubject is the four fields the sources actually read, and nothing else. Principal.subject answers one, so the run path is unchanged and the tenancy still has one writer. ContextSubject.of_actor() answers one for a turn.

A source that needs a field the turn cannot supply must say so. MemoryContextSource filters on definition_id, because a memory belongs to the agent that wrote it, so it retrieves nothing for a null definition rather than widening the read.

class ContextPacker:
    def pack(
        self,
        items: list[ContextItem],
        policy: ContextPolicy,
        *,
        failed_sources: Sequence[str] = (),
    ) -> ContextBrief: ...

The subject travels in the request, not in the spec. A ContextPolicy is authored by an admin and frozen in the run snapshot. If a source read its tenancy filter from the spec, an author could widen it. Every source therefore filters through request.subject.

The spec fields are bounded at publish time. token_budget and total_token_budget are non-negative strict integers. A negative budget names an amount the packer cannot spend. limit is a non-negative strict integer up to MAX_CONTEXT_ITEM_LIMIT, which is MAX_ROWS - 1, or 999. A source reads its rows in one PostgREST request, and PostgREST truncates a read at MAX_ROWS and reports the truncation only in Content-Range. A cap at that bound therefore asks for rows the read cannot answer. The ceiling is one constant in services/context/models.py, and no source repeats the number. A zero budget, a zero limit and None stay parseable. Admission is a separate rule: the publish check rejects an enabled source under a zero total budget, and it rejects an enabled source that holds a zero token_budget of its own. A disabled source may hold a zero budget, because it reads nothing.

options is validated, not trusted. An untyped dictionary guarded only by a sentence is the one hole in an otherwise closed tenancy story. Each source declares an options schema. ContextSourceRegistry validates the options when the definition is published.

class ContextSource(Protocol):
    name: str
    options_schema: type[BaseModel]   # KnowledgeOptions(include_platform: bool = False, include_organization: bool = False)
    async def retrieve(self, request: ContextRequest, spec: ContextSourceSpec) -> list[ContextItem]: ...

An unknown key fails the publish. A wrong type also fails without coercion.

No source schema declares an organization, a user, or a row filter. A policy that names one fails. The check costs nothing at run time.

A policy names each source once. Duplicate names would run one source twice and overwrite the accounting maps. The publish rejects them.

The registry also accepts each source name once. A duplicate declaration fails registry construction. It never replaces the first source by insertion order.

Source failure

A source failure is a context decision, not a policy decision.

• required: true and the source fails: the run fails with a clear error.
• required: false and the source fails: the name goes into failed_sources, the brief says the source was unavailable, and the run continues.

Silence is the one unacceptable answer. An agent that reasons over a silently empty CRM read looks confident and is wrong.

DefaultContextBuilder schedules each enabled source before it waits for the combined results. The default source timeout is one second, and the builder applies it independently to each read. The timeout must be a positive, finite number. The constructor raises ValueError for zero, a negative value, NaN, or infinity. This timeout is an operational guard, not a policy field and not the latency target. A disabled source is not resolved and is not read.

asyncio.gather returns the results in policy order, even when the reads finish in another order. The packer therefore receives one deterministic item order.

An exception, a timeout, or a source that is missing from the process registry is a source failure. A source also fails when one of its items names another source. Without that check, the packer drops the item because it has no budget, and the brief reports no failure.

The builder collects all source results before it decides. If one or more required sources failed, it raises RequiredContextSourceError before the model runs. The error names each failed required source in policy order. The Inngest segment step can retry the error. If all attempts fail, the run failure message keeps the source names.

An outer cancellation is not a source failure. CancelledError stops the builder and cancels the reads that are still active.

When the brief is built

The snapshot freezes the ContextPolicy. The content is always fresh.

segment 0        build the brief, and put it in the system block
later segment    rebuild only when refresh_on_resume is true and a wait happened

A rebuilt brief replaces the system block. It is never appended. An agent continues the same Agno session across a segment, so the earlier brief is still in the message history. Appending a second brief gives the model two versions of the same CRM row, with no way to tell which is current. Replacement keeps one truth in the window.

The builder does not do the replacing. It returns a ContextBrief, the brief travels in AgentExecutionRequest, and AgnoAgentRuntime composes it into the instructions it hands the framework. This layer never opens the stored session, because that would need Agno's message shape and put it above the boundary that keeps the framework replaceable. See runtime execution.

Replacement needs no surgery, and it needs the caller to supply the block every time. Agno rebuilds the system message from instructions on every run, and it never replays the stored one. That is what makes replacement free. It is also what makes a missing brief silent: pass instructions without one and the block is simply gone, with nothing failing and an agent that has forgotten the rows it was shown. The runtime therefore stores the block it composed beside the session, and context = None means reuse that block rather than trust the framework to keep it.

After a long approval wait the world has moved, so a refresh is right.

A replay of a completed segment uses the memoized segment result and does not run the builder. A retry of a failed segment can run the builder again because the segment did not commit. This can also happen for segment 0. The sources are read only, and the agent session is not written on the failed attempt. Adding a second context checkpoint or store would add durable state only to avoid this bounded read.

Tenancy

The API uses the service key. RLS does not filter a service-key read.

• A public-schema source calls scoped_db(request.subject.organization_id). OrgScopedClient adds the organization filter for tables in ORG_SCOPED_TABLES.
• An agent-schema source filters by organization_id on the owning row, or through a joined owner such as agent.knowledge_sources.
• A source never uses the admin client for an organization-scoped read.
• A context policy never carries an organization, user, or row filter.

The knowledge match function stays SECURITY INVOKER, so a future user-key caller remains subject to RLS. Its query also filters the organization explicitly. The explicit filter protects the service-key path.

Platform rows have organization_id IS NULL. The query includes them only when include_platform is true. Organization rows are filtered by request.subject.organization_id and join only when include_organization is true. The Front Door policy enables platform knowledge only.

knowledge_search(
  p_organization_id uuid,
  p_query_embedding vector,
  p_include_platform boolean,
  p_include_organization boolean,
  p_limit int
) -- SECURITY INVOKER

An unfiltered vector query is never acceptable. SECURITY DEFINER is also not acceptable.

Application state

ApplicationContextSource resolves request.entities. It never searches request.query. A search is task work and uses a tool.

Run input can carry refs, a list of ResourceRef shaped objects. The executor keeps valid objects and drops malformed entries from ContextRequest.entities. It does not change the stored run input.

The source supports crm.company, crm.person, crm.deal, crm.activity and prospect. CRM projections contain scalar fields from the named row only. Deal and activity projections include owner, state, timing and relationship ids, but they do not join the related company, person, communication or activity history. A prospect ref names public.prospects.id. Its projection contains review_state, opportunity_score, opportunity_reason, recommended_action, people_state and people_state_reason. It does not join global Intelligence rows. The table has no label field, so the resolved ref keeps the input label when one exists.

The source removes duplicate refs by (kind, id) and keeps the first request occurrence, even when a later ref has another label. It drops unsupported refs and ignores a ref that does not resolve for the subject. It batches each supported kind in one query and restores the request order after the database read.

Every public-schema read uses scoped_db. Soft-deleted CRM rows exclude deleted_at rows. public.prospects and public.crm_activities have no soft-delete column, so their resolvers do not add that filter.

Each non-null projected field becomes one ContextItem. The row ID stays in ContextItem.ref, not in a second field item. The exact column name forms the subject key. spec.limit caps the returned items in ref order and projection order. A zero limit returns no item and does not read the database. A negative limit fails policy validation. None means no item-count cap; the token budgets still cap the packed brief.

Knowledge

A file enters through an admin upload, never through a conversation. An organization document uploads to the ordinary upload API and becomes an organization scoped knowledge source. A file dropped into a chat does not: Channel Gateway counts the attachment, drops it, and answers once that the platform reads text only.

That holds for web chat too. A conversation scoped knowledge source is deferred, so V1 has one ingestion path and one retention rule instead of two.

Platform documentation is shared knowledge, not organization memory. That is what lets the front door answer "what does lifecycle mean in CRM?" without a product manual inside its prompt.

Tables

agent.knowledge_sources
  id, organization_id (null = platform), kind, title, uri, object_key,
  status: pending | ready | failed, updated_at

agent.knowledge_chunks
  id, source_id, ordinal, text, tokens,
  embedding vector(N), embedding_model

Ingestion

upload or sync -> extract text -> chunk -> embed -> insert chunks -> mark the source ready

Only a ready source is searched. A half embedded document that answers half a question is worse than no answer.

A source stuck in pending needs an owner, or it is invisible for ever. An embedding job that dies leaves a document an admin uploaded, believes is searchable, and never sees in a result. Nothing fails, and nothing alerts.

pending for longer than 1 hour  ->  status = failed, with the reason
                               ->  the connections UI shows it, and the admin retries

It is one more clause in the same scheduled sweep as the run reaper, and it is the same principle: a silent no-op looks exactly like a working system.

Pin the embedding model on the row. One query must never mix two models, because their vectors are not comparable.

One setting says which model is live. Without it, "switch the search" has no mechanism: the search would have to guess which of two model families to embed the query with, and a guess here returns plausible nonsense rather than an error.

knowledge_settings
  organization_id         null = the platform default
  active_embedding_model  every search embeds its query with this, and filters chunks to it

  UNIQUE NULLS NOT DISTINCT (organization_id)

The organization cannot be the primary key here. A primary key is NOT NULL, and the platform default row is the one that needs organization_id to be null. Postgres 15 added UNIQUE NULLS NOT DISTINCT, which treats two nulls as equal and so allows exactly one platform row. A plain UNIQUE would allow any number of them, and the search would pick a model by chance.

To change model:

1  write the new chunks under the new embedding_model; the old ones keep serving
2  wait until every source is ready under the new model
3  flip active_embedding_model                      <- one row, one instant
4  delete the old rows

Step 3 is the switch, and it is atomic because it is one column. Storage carries both sets between steps 1 and 4, which is the cost of not breaking retrieval. This is the operation that breaks retrieval quietly, so make it explicit.

Memory

ENG-2180 adds the durable store and the two write tools. ENG-2181 adds MemoryContextSource and reads stored memory into a brief. The write unit does not make stored memory available to a prompt on its own.

Working memory and episodic memory already have a source. Do not build a store for either.

agent.memories
  id, organization_id, definition_id, scope: user | organization,
  user_id, subject_key, text, subject_refs jsonb,
  proposed_by_run_id, supersedes_id, forgotten_at, created_at

agent.memory_heads
  the current revision of each memory chain, including a tombstone

agent.live_memories
  the current non-tombstone revision of each memory chain

definition_id is the agent definition that owns the memory. An organization memory is shared by the users of that definition. A user memory is visible only to that user in that definition. A trigger has no user, so it can write an organization memory only.

V1 stores zero or one entry in subject_refs. With one ref, the service creates the key from that ref and the attribute. With no ref, it creates the key from the scope owner and the attribute: organization:<id>:<attribute> or user:<id>:<attribute>. A list with two subjects would make the service choose one without a rule, so it is refused. The storage stays a JSON array so a later contract can add a relation key without a table rewrite.

The table and both views are closed to authenticated. Writers read agent.memory_heads on the service role. MemoryContextSource reads agent.live_memories on the service role and applies the subject filters. A broad RLS policy on an organization memory would expose its text before the source resolves the subject reference through the owning domain.

The write path is a tool

An agent writes memory by calling memory.remember, which is an ordinary write tool.

Only an agent definition can own memory. A workflow tool node cannot call memory.remember or memory.forget; publish validation refuses that node.

agent -> memory.remember -> ToolInvoker -> policy -> MemoryService -> agent.memories

This replaces the separate propose and commit services. The gate a proposal needed is exactly what policy already is: it can allow, deny, or require a person. The idempotency claim, the span and the count limit come with it. A second pipeline beside the tool layer adds a parallel approval path and a parallel audit trail, and answers nothing new.

class MemoryService:
    async def remember(self, inv: ToolInvocation, candidate: MemoryCandidate) -> AgentMemory: ...
    async def forget(self, inv: ToolInvocation, memory_id: UUID) -> AgentMemory: ...

MemoryService refuses a write when:

• the attribute does not use the observation. or preference. memory namespace;
• a trigger asks for user scope;
• the text duplicates a live memory with the same subject_key.

Duplicate and supersede are the same check with two answers, so state which fires. Identical text under a live subject_key is a refusal, because nothing changed and a second row would only split the audit trail. Different text under that key is a supersede, because the fact moved.

Revisions are append only

A new memory with the same subject_key points back with supersedes_id. "Write in British English" then "write in American English" produces a chain of two rows. agent.live_memories returns the second row alone.

The chain has two database constraints. One chain has one root, and one revision has one child. The self-reference also carries the chain identity, so a child cannot change the organization, the definition, the scope, the user or the subject key.

root revision <- changed revision <- tombstone

memory.forget appends the tombstone. The tombstone has no text and carries forgotten_at, so the live view returns no row for that chain. A later remember can append after the tombstone and make the memory live again. Both writers read the exact head from agent.memory_heads. They do not infer lineage from a time sort because two revisions can have the same timestamp.

memory.forget accepts a current live memory id only. It refuses an old revision or a tombstone. The service checks the invoking principal before it appends the tombstone.

Each state transition is one insert. Two writers can read the same head, but only one can attach to it. The loser reloads the new head. Identical text then answers a refusal. Different text appends after that head. A worker crash can leave a complete revision or no revision, and never half a supersede.

The tool span and the idempotency journal already record the call. The revision chain records the memory history. memory.remember is not metered, so it writes no usage row.

The guard is exactly as good as the key, and no better. Two observations of one fact filed under two keys both survive, and the model reads both. MemoryService creates the key from subject_refs and the attribute rather than taking a free string from the model, which removes the common case. It does not remove the hard one, and V1 does not add a model call to find near duplicates. Say what it does, and do not claim more.

The input gate accepts attributes in the observation. and preference. namespaces only. Application sources use their field names without either prefix. This refuses a direct memory write to an application-owned attribute. It cannot detect that free text restates the same fact under another attribute.

A model cannot supply subject_key. The strict input model rejects that extra field as invalid_input; it never ignores an argument in silence.

A model never writes agent.memories directly, and never through a repository.

Reading memory

MemoryContextSource returns agent.live_memories rows for the organization, the user and the definition. It ranks them by recency alone. Phase 3 has no relevance signal, because embeddings stay out of scope.

The order is (created_at DESC, id DESC). Two revisions can carry one timestamp, so the id breaks the tie and keeps the cap deterministic. A trigger run reads organization scope alone: an organization row stores a null user, so a user filter would exclude every row.

spec.limit caps the database read. A zero limit returns no item and does not read the database. None reads 20, because memory has no entity list to bound it, unlike application state.

The view drops scope_owner_id and forgotten_at, so AgentMemory cannot validate a live row. The read path declares its own row shape.

An organization memory that names an entity is filtered by that entity. A memory row keeps subject_refs, and the owning resolver applies its tenant guard. When a ref does not resolve for this subject, the item is dropped.

A model supplies subject_refs, and the write path stores the value without resolving it. The guard therefore drops a ref that names another tenant, a deleted row, a row that never existed, and a kind no resolver owns.

⚠️ The guard is a tenant guard, not a per-user guard. scoped_db filters by organization, and the CRM read policy grants each member of an organization the same rows. Two users of one organization therefore read the same memories. The guard inherits a narrower CRM rule the day one exists. Until then, do not claim it separates two people of one tenant.

The source resolves the distinct subjects in one query for each ref kind. It never reads one query for each memory. A ref list splits into batches of 100, because no source bounds its ref count and a long URL is answered 414.

V1 resolves one subject for each memory. MemoryCandidate and a table CHECK both bound the list to one entry, and the read path refuses a second entry rather than rest on either.

The read path fails closed. MemorySubjectRef accepts any kind string, so a stored ref can name a kind no resolver owns. That ref resolves to nothing and the memory is dropped, which makes such a write a memory nobody reads.

A malformed stored element costs its own memory, never the page. The table checks the array and not the element, so a row can hold an element with no id. parse_stored_refs reads each element, ignores a key it does not know, and counts what it refuses. The source then drops that one memory and logs the drop, because an unfiltered memory would otherwise reach every member of the tenant. The write path keeps the row instead, so memory.remember and memory.forget still serve the chain. A CHECK constraint stops a new row of that shape. The parser stays, because a read must not wait for the constraint to reach every environment.

One element is well formed when all three validators say so. kind and id join the subject key, so each one follows the rules MemorySubjectKeyPart states: a non-blank string that holds no : and no null byte. The CHECK and parse_stored_refs state the same two rules, and jsonb refuses a null byte on its own. Blank means every blank character, not the space alone. The CHECK therefore reads ~ '[^[:space:]]'; btrim removes the space and nothing else, so a tab or a no-break space would pass the write and fail every later read. label joins no key, so it stays free text and may hold a colon. The three must agree, because _answer rebuilds MemorySubjectRef from a stored element: a part the parser accepts and the model type refuses raises inside a tool result instead of dropping the one memory.

The guard drops a memory after the cap selected it, and no second query refills the answer. The result can hold fewer rows than the cap. A refill would spend a query on rows the packer may cut anyway.

Run history

Run history is the retrieval mechanism for episodic memory. It is not a fourth authority.

It reads agent.runs once. The query has these filters:

• the organization equals request.subject.organization_id;
• created_at is inside lookback_hours, which defaults to 24 and accepts 1 through 720;
• the row belongs to the same user or trigger, or its result refs contain an entity in request.entities by kind and id;
• the row id is not request.run_id, when the request names a current run.

The query reads root runs and child runs. Either run can produce an entity. It orders the rows by (created_at DESC, id DESC). It applies spec.limit, or a default of 5 when the limit is None. A zero limit returns no item and does not read the database. ContextSourceSpec rejects a negative limit. The source adds no second item-cap option.

ContextItem.body names the run status first. It then names the parent run, the valid result summary and the result refs, and it ends with the waiting reason or the failure reason when the status is one of those. title is Run <status>. ref points at the run, and recorded_at is the run creation time. The item mints no subject_key.

Result refs select relevant runs, and the item body also names them. The body is the one field _prompt reads, so a request about the rows of an earlier run needs the ids there. The body renders each ref as <kind> <id>, and kind is the word the capability input accepts, not the word the tool stored. capability_ref_kind in src/agentic/shared/refs.py owns that map and states which kinds it joins (ENG-2345).

The body authorizes nothing. ContextItem.ref stays the run, so ContextBrief.items holds run refs alone and a reply can cite no row of a result. Ref matching and deduplication use kind and id; they never use the optional label, and a stored ref carries no label.

The organization is the current run-read boundary. It is the same boundary that the run explorer applies, so the run ref resolves for this subject. The actor and entity tests select relevant rows; they do not grant access. A future narrower run-read rule must apply here and in the run explorer together.

The query selects only id, status, result and created_at. It never returns a span tree, input, principal or snapshot into a prompt. It adds no table or rollup.

Front door context

The front door uses this same builder with one fixed platform policy named front_door, and it gets no builder of its own. It must not run a fresh exploratory CRM, email, web or research search, because that is task work, and task work is delegated.

Front door owns the policy values and the list of sources it allows.

Reuse isolated application readers

The CRM domain services import the legacy agent stack through their current dependency graph. src.agentic cannot import those services while the isolation contract is active.

An application source and an internal tool handler can share projection models and pure filters inside src.agentic. Both read public CRM tables through scoped_db. Neither uses the admin client.

ApplicationContextSource ─┐
                          ├-> shared projection -> scoped_db -> Postgres
CRM tool handler ─────────┘

The builder reads state as infrastructure. The same tenant filter, soft-delete rule, and projection still apply.

Budgets and metadata

Token counts are estimated with a cheap heuristic, and the total budget is a hard cap. Precision is not worth a tokenizer call per item.

The estimate is one token for each four bytes of the UTF-8 rendered block. The count is bytes, not characters: one CJK character costs three bytes and one emoji costs four, so a character count under-reports those scripts by three to four times. It is never zero for a block that has text. A zero estimate would let an unlimited number of short items pass a budget that never binds.

The packer spends in one pass, and it charges four costs to the total:

1. the section heading, once, when the first item of that class is admitted;
2. each admitted item, against the total and against its own source budget;
3. nothing for a dropped item;
4. the unavailable-source line, before any item, because a run must never read a silently short brief.

The unavailable-source line is the one cost the total does not cap. Every item stays inside the total, and inside its own source budget. The notice does not, because a brief that omits it makes the run reason over a gap it cannot see. ContextBrief.tokens reports the true cost either way.

The packer drops a whole item. It never truncates a body. A half fact reads as a whole fact, and the model cannot see the cut. ContextBrief.dropped is an item count, so a partial item would have no honest value to report.

ContextItem.tokens states what a source already measured. The packer uses it when it is above zero, and it estimates the block when it is zero.

The packer renders each item on one line. It collapses every run of whitespace in a title and a body to one space. A source supplies extracted document text and agent-written memory text. A body that kept its line breaks could start a line with ## , and that line forges a section that outranks the section it sits in. The collapse covers every whitespace character, not the newline alone: a carriage return, a form feed, and the Unicode line separators forge the same heading. A source name in failed_sources is normalised the same way. A source returns data, and only the packer writes the section structure.

A subject_key is never the empty string. An empty key is no key, and it must not gather every keyless item into one collision group.

The packer applies precedence before it spends. A lower-class item that loses a subject key must not first eat budget that a higher-class item then cannot use.

Two items of one class can share one complete key. The first item in the pack order stays, and the second is counted as dropped. The packer sorts by class order and then by policy source order, so a concurrent builder that answers out of order still packs one brief. One source keeps the order it returned. The order covers the enabled sources only.

per_source_tokens and dropped carry one entry for every source that appears in items. A source that returned no item appears in neither map. failed_sources is the record of a source that did not answer.

An item whose source no enabled spec names is dropped and counted, before precedence runs. The packer never spends a budget it was not given, and an item it will drop must never win a subject key first. A disabled source is dropped by the same rule, whatever its policy index.

ContextBrief reports the pressure without storing the prompt:

tokens, per_source_tokens, dropped counts, failed_sources, item refs

The full prompt is not stored by default. See observability and operations.

Latency target

Independent sources are read concurrently. These are design targets to confirm in production.

The default one-second source timeout is an outage ceiling. It is not an acceptable steady-state duration. Optimize a slow source. Never turn retrieval into a second agent hop.

Physical storage

Scenarios that shaped this design

Rules

• Context construction is deterministic and model free.
• The subject travels in the request. A policy never carries a tenancy filter.
• Application state resolves explicit refs only. It never searches the request query.
• Public CRM reads use scoped_db with the subject organization. The service key bypasses RLS.
• Independent sources are read concurrently.
• Each enabled source has an independent timeout. The default is one second.
• Concurrent results keep policy order, and cancellation stops the build.
• A source returns structured items. Only the packer writes prose.
• Precedence is section order, budget order, and same subject wins upward.
• Precedence covers all six kinds, and run history ranks last.
• One row names the live embedding model per organization, and exactly one names the platform default.
• A required source failure fails the run. An optional one is reported, never hidden.
• An item that names the wrong source fails that source, so it never disappears silently.
• Platform documents use an explicit shared scope. Organization documents stay tenant scoped.
• Only a ready knowledge source is searched, and the embedding model is pinned per chunk.
• The live web is a tool, not a store.
• Memory is written through the memory.remember tool, and revisions are append only.
• A memory attribute uses observation. or preference., never an application field name.
• Memory ranks by recency alone in V1, and a subject ref no resolver owns drops the memory.
• The memory subject guard is a tenant guard. Per-user CRM visibility does not exist yet.
• One chain has one root, one child per revision and one current head.
• A tombstone forgets a memory without making an old revision current again.
• A rebuilt brief replaces the system block, and AgnoAgentRuntime is what replaces it.
• A subject_key is <entity kind>:<entity id>:<attribute>. make_subject_key() creates it from one subject reference, or from the scope owner when there is no reference.
• A policy and the registry each accept one entry per source name.
• A policy that enables a source states a total token budget above zero.
• The packer drops a whole item, and it never truncates a body.
• The packer renders one item on one line, so no body can forge a section heading.
• The token estimate counts UTF-8 bytes, never characters.
• One setting names the live embedding model, and a search filters chunks to it.
• A source pending for more than an hour is failed, never left invisible.
• No new store because a concept needs retrieval.

Open decisions

1. Does memory retrieval need embeddings in the first release, or are scope, recency and subject_key enough?
2. How large may the front door knowledge budget be per turn before the turn feels slow? Decided for the Front Door: 1,500 tokens across up to eight chunks.
3. When conversation attachments arrive, is a chat file embedded by default, or only when a person asks a question about it?

Minimum contract tests

• A context policy naming an undeclared source option fails to publish.
• A context policy naming an option with the wrong type fails without coercion.
• A context policy naming one source twice fails to publish.
• Registry construction refuses two sources with one name.
• An organization A vector search cannot return organization B chunks.
• Platform documents are reachable without weakening a tenant rule.
• A context policy cannot widen the organization or user scope.
• Malformed run-input refs are dropped, and valid refs reach ContextRequest.entities.
• Application state reads only explicit, distinct, supported refs.
• Two refs with the same (kind, id) and different labels keep the first ref.
• A missing, deleted or cross-organization CRM ref returns no item.
• Application state uses at most one subject-scoped query per supported kind.
• The application item cap keeps deterministic ref and projection order.
• A zero item cap reads no row, a negative cap fails validation, and None applies no item-count cap.
• A higher context class beats a lower class when two items have the same complete key.
• Two items of one class with one key keep the first, and count the second.
• The pack stays inside the total and per source budgets, and the section headings are charged.
• A policy that enables a source under a zero total budget fails to publish.
• An enabled source with a zero token_budget fails to publish, at its own path. A disabled one publishes.
• An item whose source the policy does not enable is dropped before it can win a subject key.
• A disabled source that sits first in the policy never displaces an enabled one.
• An item body that holds a section heading cannot open a second section.
• The pack order does not depend on the order the builder answered.
• A brief names every unavailable optional source before its first item.
• An optional source failure lands in failed_sources, and the run continues.
• A required source failure fails the run.
• A blocked source does not stop another source from starting, and results keep policy order.
• An optional timeout and a missing optional source both land in failed_sources.
• A zero, negative, NaN, or infinite source timeout fails construction.
• Every failed required source is named in one error.
• A disabled source is never resolved or read.
• A source item that names another source fails its source.
• Cancelling the builder cancels active reads and does not create a brief.
• An empty or all-disabled policy returns an empty brief.
• A segment after approval holds one brief, not two.
• A model cannot write agent.memories except through memory.remember.
• An old revision and a tombstone never reach a brief.
• A memory naming an entity of another tenant is dropped for that subject.
• A knowledge query never mixes two embedding models.
• An object-backed knowledge source has the same per-organization idempotency boundary as a URI-backed source.
• A chunk inherits tenancy from its source rather than storing a second organization field.
• An application field and a namespaced memory attribute do not create the same key.
• Missing subject data creates no key. Empty or colon-delimited key parts fail.
• A memory about a company's buying process is not dropped by a CRM row about its lifecycle.
• A completed segment replay rebuilds no brief. A failed segment retry can rebuild one.
• A segment that resumes from a wait rebuilds one when the policy asks.
• A source stuck pending is failed within the hour, and the admin can see why.
• A run history item creates no subject_key.
• A memory whose stored ref names a kind no resolver owns is dropped.
• One memory read resolves each ref kind once, never one query for each row.
• A trigger run reads organization memory alone.
• A second platform row in knowledge_settings is refused by the database.
• Re-observing unchanged text writes no second memory row.
• Concurrent changes form one linear chain with one current head.
• Forgetting a memory appends one tombstone and exposes no older revision.