AgencyCore Docs

00

Why push

problem

Static company data goes stale fast (B2B decay ~2.1% / month). The moment a brand starts running Meta ads, posts a Head-of-Growth role, or closes a Series A is a finite window to win the conversation. Today we only surface this when a user runs a Sonar / Headhunter search - pull-based and rare.

2.1%

Monthly B2B data decay

why facts need watching

off→on

Transitions, not states

slow→fast counts too

auto

Routed to the right inbox

no manual search

GOALCatch the transition, route it automatically

A push layer that detects signal transitions and routes them to the matching agency inbox with no human in the loop. Sibling of the data graph - facts live in the intelligence databases, this watches them change.

01

System shape

two planes

A data plane (collectors writing append-only events) and a match plane (each agency's ICP rule set, evaluated per event). They meet at signal_events; survivors land in a per-org signal_inbox.

data plane public sources → collectors

match plane per-agency ICP rules

signal_events (append)

company_id · kind · payload · severity · observed_at

↓ scoring agent: reasons over each org ICP → score · rationale · draft

signal_inbox (org · RLS)

org_id · event_id · score · status

Sonar inbox · SSEEmail digestSlack / webhook

Data plane = data graph + the new signal_events table. Match plane = each agency's ICP rule set evaluated per event. One event fans out to N inbox rows, one per matching subscription.

02

System architecture

C4 · containers

The same two planes as concrete, deployable containers - one colour per repo. The data path runs top to bottom; each edge is labeled with what crosses it and how. Solid edges are request-path / synchronous; dashed edges are scheduled or async. Hover any container for what it owns, or trace one event down the happy path.

ac-monitorac-backendmanaged (Anthropic)ac-python-apiac-frontendexternalsyncasync

External systems

HTTPS poll · celery beat 15m → weekly · diff vs last snapshot

Collection

INSERT · dedup_key gate (company|kind|event_key, type-specific)

Event log + rules (Postgres)

new event → api starts a managed-agent session (event + company facts + ICP)

Match plane — scoring agent

survivor → fan out N concurrent enrich pipelines (one per surviving signal)

Enrichment fan-out — complete the pair

UPSERT intel_companies + intel_people · provenance · dedup · cache-first

Intelligence write-back (Postgres)

managed session → SSE event stream (assistant turns · tool_use · tool_result)

Event relay (the SSE seam)

INSERT · one row per matching org (never twice - UNIQUE(org,event))

Delivery store (Postgres)

pg LISTEN inbox.insert + relay agent.progress → 1 consumer → pub/sub

Realtime fan-out

EventSource (SSE) · upsert by inbox.id · paginated REST for history

Surfaces

Six repos now: ac-monitor collects, ac-backend owns the signal tables and the intel_* write-back, the Claude managed agent scores on Anthropic infra, ac-python-api runs the enrichment fan-out + relay + delivery, ac-frontend renders the inbox. The new middle tiers are the heart: on a survivor the agent completes the pair (enrich_company → Headhunter DM → enrich_person), each resolved fact is written back to intel_* with provenance, and the fills stream to the open tab as agent.progress / agent.done frames - see §03. Email and Slack are views over signal_inbox; the inbox row is the single source of truth.

03

Stream contract

two SSE legs

Two SSE legs meet at the ac-python-api relay. Upstream we subscribe to the managed agent's hosted session stream; downstream we emit our own SSE to the browser - the same frame contract the agent-run inbox already speaks. The relay is the only place the two touch.

upstream · we subscribe

Claude managed agent (hosted session)SSE ↓ac-python-api · event relay

Anthropic session SSE

assistant turn · tool_use · tool_result · message_stop

The hosted loop runs on Anthropic infra. We never see the loop - only the events it streams. The relay answers tool_use by running the enrichment fan-out (enrich_company / enrich_person / Headhunter) and posting tool_result back into the same session, and mirrors the user-visible subset into Supabase for audit.

downstream · we emit

ac-python-api · SSE fan-outSSE ↓ac-frontend · EventSource

our SSE (contract unchanged)

inbox.insert · inbox.update · agent.progress · agent.done

One LISTEN consumer + the relay's in-process agent.progress feed merge into a pub/sub fanned to N browser tabs. The frontend keys every frame on inbox.id and upserts - inserts prepend a card, updates patch it in place. Same frame contract the agent-run inbox already speaks.

The relay answers tool_use by running the enrichment fan-out and posting tool_result back into the live session, mirrors the user-visible subset into Supabase, and writes the signal_inbox row. Everything downstream reads from that row - so a managed-agent run and a plain deterministic insert reach the tab through the identical frame schema.

Downstream frame schema what each frame does to the Pinia store

inbox.insert{ id, company, kind, score, status:"new" }Prepend a new card (or no-op if id already present - replay-safe).

agent.progress{ id, phase:"enriching", note }Patch the card to a live "enriching…" state while the agent works.

agent.done{ id, rationale, draft }Fill in the rationale + drafted outreach on the existing card.

inbox.update{ id, status:"read"|"dismissed"|"added" }Patch status in place - a write from any surface or tab lands here.

Lifecycle invariants the rules the frontend leans on

upsert by idStore is a Map keyed on inbox.id - every frame upserts, never appends. Reconnect + replay is idempotent.

Last-Event-IDEach frame carries a monotonic id; on reconnect EventSource replays the gap from the inbox cursor (delivered_at, id).

id-only NOTIFYpg NOTIFY caps at 8KB - it carries inbox.id only; the SSE layer SELECTs the row (RLS-scoped) before framing.

heartbeatA :keepalive comment every < 55s holds the connection open past the Heroku router idle timeout.

authEventSource cannot set headers - token rides a cookie; org scope is enforced by RLS on the hydrate SELECT, not the client.

one consumerA single session-mode LISTEN connection fans to all tabs - never one LISTEN per EventSource (and never through txn-pooled pgbouncer).

04

Signal taxonomy

agency lens

Start narrow - ship a handful of high-signal kinds before adding more. Each kind has its own collector and dedup window. Click a kind to inspect why it is hot for agencies.

started_meta_adsseverity 4 · daily dedup

Started Meta ads

Why hot: New advertiser = budget appeared, no agency yet locked in.

Source: Meta Ad Library diff

05

ICP rule matcher

the agent's gate tool

The rules stay declarative - a JSON AST, no code per subscription. With the scoring agent in front (§02), this AST is no longer the whole engine: it is the deterministic gate tool the agent calls to bound its reasoning. Configure a sample company and its live signals; the AST evaluates leaf-by-leaf into a weighted score, normalized 0-1. Cross the threshold (default 0.6) and the agent proceeds to draft; below, it drops.

company facts

industrycountrysize_band

live signals

started_meta_ads age 6d

hired_growth_role title

match threshold 0.60

all (every clause)

✓company.industryin ["apparel","beauty","homegoods"]apparel

✓company.countryin ["US","CA","GB"]US

✓company.size_bandin ["11-50","51-200"]11-50

any (at least one) ✓

✓started_meta_adswithin_days 146d ago

✓hired_growth_roletitle_contains ["growth","performance","paid"]"growth"

match score

1.00/1.0

100%

Match · inbox + digest + Slack

High score and a hot signal. Surfaces in every configured channel.

Match score = weighted hit count, normalized 0-1. Weights here: industry 1.00 · country 0.80 · size 0.80 · any-signal 1.40. Per-org tuning ships later.

06

Schema additions

tables

Three new tables, all global-read / service-role write like intel_*. signal_events is global and append-only; signal_inbox is org-scoped (RLS by org_id).

global · append-only

`signal_events`

intel_company_id FK
kind text
severity int
payload jsonb
source text
observed_at ts
dedup_key UNIQUE

per agency org

`signal_subscriptions`

org_id FK
name text
icp_rules jsonb
min_severity int
channels text[]
paused bool

org-scoped · RLS

`signal_inbox`

org_id FK
signal_event_id FK
subscription_id FK
match_score numeric
rationale text
draft text
agent_session_id text
status text
delivered_at ts
UNIQUE(org,event)

UNIQUENever deliver twice

signal_inbox carries UNIQUE(org_id, signal_event_id) and signal_events a dedup_key of company|kind|event_key, where event_key is the type-specific identity (funding → stage, exec change → role + person, hiring → role-family) - not a coarse week bucket that would collapse two distinct same-week events. This is the same shared event-key the Sonar pull pipeline dedups on, so both converge on one canonical row. Many sources for one event become corroboration on that row, not duplicate inserts. One event fans out to N inbox rows, one per matching subscription - but never twice to the same org.

07

Collector cadence

celery beat

Each source polls on its own beat and diffs against the last snapshot. Per-host token bucket; failure → backoff + dead-letter. A stale collector pages oncall - missed signals are the failure mode that hurts most.

15mmeta_ad_library_poll→ diff vs last snapshot

hourlygreenhouse_lever_ashby→ diff job postings

hourlysitemap_homepage_diff→ trigger website enricher

dailycrunchbase_funding_feed→ funding rounds

dailysec_edgar_press_releases→ public-co material events

weeklycommon_crawl_backfill→ wide-net coverage

08

Delivery fan-out

inbox is truth

The inbox is the source of truth. Email and Slack are views - a dismiss in any surface writes back to signal_inbox.status.

SSE

Realtime open Sonar tab

On insert + agent progress, push inbox.insert / agent.progress / agent.done / inbox.update frames to any open tab. The frontend upserts by inbox.id. Full leg + frame schema in §03.

07:00

Daily digest sendgrid

SELECT ... WHERE status='new' AND delivered_at IS NULL at 07:00 org-local, grouped by subscription into one templated email.

opt

Slack / webhook per subscription.channels

Optional fan-out for orgs that wire it. Same inbox row, additional view.

09

Hot-signal sharing

A vs B

"If there's a hot new signal, share with agencies whose ICP matches." Two interpretations - pick one.

recommended · default

Each agency gets the signal independently

A subscription match writes a row in that org's inbox. Many agencies can receive the same event. The data graph is already global; signals are downstream of it, so this falls out for free.

Why default: no claim flow, no expiry, no billing coupling. Lowest product complexity for the clearest value.

10

Build order

ship to validate

Stop after step 3 and ship to staging with one design-partner agency. Validate signal quality before adding sources - more collectors on a noisy matcher just amplifies noise.

1

Schema + RLS

signal_events + signal_inbox (incl. rationale / draft / agent_session_id) + signal_subscriptions, pgTAP RLS, dedup_key unique constraint.

2

One collector + scoring agent end-to-end

meta_ad_library_poll → signal_events → managed-agent session → relay persists score + rationale + draft → signal_inbox row → email digest, for one hard-coded org.

3

Subscription CRUD UI ship here ✓

Sonar settings page: JSON AST rule builder (the agent's gate tool), severity threshold, channel toggles.

4

Second collector

greenhouse_lever_ashby - proves the collector abstraction generalizes.

5

SSE realtime push (both legs)

Relay re-streams agent.progress / agent.done plus LISTEN inbox.insert / inbox.update into the open Sonar tab.

6

Slack / webhook fan-out

Per-subscription channel delivery beyond inbox + email.

7

Agent + score tuning

Per-org analytics on open / dismiss / added-to-CRM, fed back into the scoring prompt and the AST gate.

11

Open questions

undecided

Subscription quota

Cap per-org subscriptions (~25) to bound matcher work? Probably yes; revisit on real usage.

Backfill window

Replay last N days of events into a new subscription's inbox. Default 7 days, configurable - useful first impression, noisy long-term.

Cold-start coverage

Ad-library polls surface companies never ingested. Auto-upsert into intel_companies from the collector so events always have a parent row.

Decay vs persistence

Keep signal_events forever (audit, ML) but expire signal_inbox rows after 90 days to bound UI noise. Confirm with product.

LEGALPublic business data only

Every source is public business data - no personal scraping. A payload naming a hire falls under GDPR Article 14: store the source URL, expose an opt-out, honor takedowns within 30 days. Filter by redistribution_allowed before composing any digest body. Each org's signal_inbox is RLS-isolated - no cross-org leak of dismissals or notes.