Workflow Engine · ac-python-api · Claude Agent SDK

Company Signals

Sonar finds companies worth contacting because of a signal — a funding round, a new CMO, a product launch — and scores each by how strongly that signal predicts marketing budget or need for the agency running the search. A deterministic pipeline with two bounded agentic nodes, a global intelligence cache, and a composite score computed in code.

Runtime Claude Agent SDKAgentic nodes discovery · enrichmentScoring deterministic compositeCache global intelligence DB
Brief + playbookDiscover · enrich · score·intelligence DBworkflow_run_companies
00

What Sonar does

End users are marketing agencies doing cold outreach. Sonar surfaces companies worth contacting because of a signal, then scores each lead by how strongly that signal correlates with marketing budget or a need the agency can address. Every search discovers signals live — they’re perishable, so Sonar never trusts a cached one. The global intelligence cache only spares re-enriching the slow-moving company facts: the first org to learn a company pays once, and every later search skips its enrichment for a fraction of the cost.

2
Bounded agentic nodes
discovery · enrichment
15
Signal types
agency-weighted taxonomy
4
Enrichment tiers
cache → cheap → exp → web
0
LLM-emitted scores
composite in code
KEYThe signal-first bet

A company is worth an agency’s attention at the moment a trigger event creates budget or a gap — not because it sits in a static list. Sonar treats intent as perishable inventory: it discovers dated signals, weights them by source trust and freshness, and re-scores against each agency’s own ICP and offer. The global cache means the expensive part — finding and verifying the signal — is paid once and reused.

01

System architecture

The whole run as a closed loop. A brief leaves the frontend, the API opens a run on the Claude Agents Platform, which discovers and resolves signals then fans out per company to the enrichment agent - the agent reads the intel cache or runs the waterfall and writes firmographics back. Results return along two paths: per-field patch frames stream live to fill the table (emit → run-SSE → apply) while resolved cells persist durably to the database. A later search skips enrichment for any company already cached. Follow the arrows, or click any stage to trace it. The frame schema and how the frontend applies each patch are in section 09.

POST /workflows/runbrief · ICP · offerkick off runhost-side toolsfan out per companyone pipeline eachresolved cellsfacts + contactper-field framesrun SSE · keyed by idpersist · paralleldurable rowsfinish · final readworkflow_run_companies1Frontendlive cell tableac-frontend2API + Sonar serviceFastAPI · workflow engineac-python-api3Discovery + scorestrategist · signals · resolveClaude Agent SDK4Enrichment agentfan-out per companyenrichment agent5Frame emitterper-field patch framesSSE · emit6Run-SSE to cell tablefrontend applies framesSSE · apply7Databasedurable write-backSupabase
02

Agent workflow

The detailed breakdown of one Sonar search as a tiered map - one colour per repo, top to bottom along the data path. Search signals from the public data sources; once found, the run fans out per company to the enrichment agent (cache-first); each enrichment writes back to the global intel DB with provenance; then per-field frames stream to the live table. Hover any container for what it owns, or trace one company down the happy path.

Claude Agent SDKac-python-apiac-backendac-frontendexternalsyncasync
Signal sources
Discovery — search the sources
Resolve + dedup (host code)
Enrichment fan-out — per company
Intelligence write-back (Postgres)
Composite score (host code)
Save-back + frame stream
Surface
03

Signal sources

Sources are a registry: each self-registers a schema, handler, category, availability check, and reputation weight at import; the discovery agent’s toolset is rebuilt per call, so a new provider appears with zero orchestration change. Discovery today is all layer 2 — Exa, Parallel, and a web_search fallback, all pull. The roadmap moves weight down the stack: free primary documents and webhook event feeds surface the event, search verifies and gap-fills it.

  1. L0
    Free primaryfilings · wires · firehose

    Primary documents at source latency, $0. Beats every paid provider on US funding and leadership latency.

  2. L1
    Structured event feedswebhooks · pre-categorized

    Events arrive pushed and typed — no polling, no extraction. The biggest gap in the current stack.

  3. L2
    Neural searchExa · Parallel — in prod

    Semantic discovery, plus verification and gap-fill of everything the layers above surface. Today this layer is the whole stack.

  4. L3
    Web fallbackweb_search — last resort

    Long-tail grounding. Cited and verified before storage, stored low-reputation.

KEYPush beats poll

Every winning signal product layers a structured event feed under a search layer. A webhook delivers the event once, typed and dated, the moment it happens; a search API has to be asked, per query, and the agent pays tokens to extract what a feed would have handed over as JSON. Search stays essential — for verification, corroboration, and the long tail no feed covers.

Which source sees which signal first? Click a signal type to rank its best sources, click a row for detail, and toggle sources in and out to price the stack.

sourcefreshcost
L0 · Free primary
addfeedhoursfree · 10 req/s
addfeedminutesfree
addfeed15 minfree · 250/query
L1 · Structured event feeds
addfeedreal-timePAYG · from $40/mo
addfeed< 24 hfrom $59/mo
L2 · Neural search
livesearchhours · livecrawl$7 / 1k queries
livesearchlive web$5 / 1k queries
addsearchlive web$5 / 1k · 4k free
L3 · Web fallback
livesearchlivetoken-metered
Intent stays unsourced — deliberately. The taxonomy’s intent-surge signal has no wired or candidate source above: the intent vendors (Bombora, 6sense, Demandbase) sell topic surges, not discrete events, on enterprise annual contracts (~$25k+/yr) — wrong signal shape and wrong cost shape for this stack. Revisit at enterprise GTM scale.
04

Signal taxonomy

v1 seed weights, tuned to a marketing agency’s value: signals that imply fresh budget or a fillable gap rank highest. Pick the agency’s offer and watch the weights re-rank — the offer conditions which signals matter.

funding 1.00

Fresh capital = budget for demand gen, brand, content. Scaled by stage / $.

leadership 0.95

New CMO / VP marketing = fresh mandate to spend and reshape the team.

hiring 0.90

Open marketing roles = a team gap the agency can fill.

intent surge 0.85

3rd-party topic-intent surge — strongest direct in-market signal.

growth / launch 0.80

New market or product = GTM and campaign needs.

M&A 0.75

Drives rebrand, integration, and budget — high for brand / creative.

ad-spend 0.70

Detected new campaigns or spend increase — strong for performance shops.

rebrand 0.70

Active marketing investment in progress; a warm door for creative / web.

content 0.60

Published pain-point content = demonstrated need; warm entry for SEO.

martech migration 0.60

Tool switch = integration / replacement work for martech / ops.

competitor switch 0.55

Using or leaving a competing tool = a switching window.

layoffs 0.50

In-house cut can open an outsourcing door — boosted for staff-aug.

partnership 0.50

Co-marketing need around a new partnership or channel launch.

award / PR 0.40

Momentum signal, softer; a tiebreaker or warm-up.

compliance 0.40

Regulatory change = specialized content for regulated verticals.

05

Dedup, corroboration & identity

One real-world event surfaces from many sources — a Series A lands in SEC, Crunchbase, a trade outlet, and the company blog. That is not noise to discard; it is corroboration. Sonar folds the sources onto one canonical signal and keeps every source as evidence, so confidence and provenance survive dedup.

Fold the stream — interactive

Watch raw observations collapse onto canonical signals

Seven raw observations arrive from different sources. Step through them: each opens a new event, corroborates an existing one, or drops as a duplicate. The same key collapses them; independent sources lift confidence, a syndicated repost does not.

0 / 7 processed
raw observations
SEC EDGAR
acme.com · funding · Series A · $12M
acme · funding · A
Crunchbase
acme.com · funding · Series A · $12M
acme · funding · A
TradePress
acme.com · funding · Series A
acme · funding · A
Newswire repost
acme.com · funding · Series A (syndicated)
acme · funding · A
SEC EDGAR
acme.com · funding · Series A · $12M
acme · funding · A
Greenhouse
acme.com · hiring · VP Growth
acme · hiring · growth
Lever
acme.com · hiring · VP Growth (same req)
acme · hiring · growth
canonical signals

Press Play — observations fold in here, one per step.

new eventcorroborated +syndicated · no boostexact dup · dropped
The model

Canonical signal + N observations

One signal row per real-world event (company · type · event-key), with many observation rows beneath it — one per source, each carrying url, observed_at, and the extracted payload. "Three outlets → one funding round" holds at the signal level; the three sources live on as evidence.

Why keep the sources

Provenance + confidence

Corroboration feeds the score (section 07), audit needs every source URL, and the push system’s GDPR Article 14 duty requires the source on record. Collapsing to one anonymous row throws all three away.

Identity is type-specific, not a time bucket

A coarse company|type|week key collapses two distinct same-week hires and splits an event covered across a week boundary. The event key is extracted per type, then used to match:

fundingcompany · stage+ amount bucket; seed ≠ Series A.
exec changecompany · role · personTwo different VPs are two signals.
hiringcompany · role-familyOne req reposted across Greenhouse / Lever / Ashby = one signal.
ad-spendcompany · platform · windowCampaign window, not per-impression.

Match pipeline: resolve company (domain + alias table) → extract event key → block by (company, type) → match exact + fuzzy → merge + recompute confidence. Blocking keeps it out of O(n²).

TRAPCount independent sources, not reposts

Five blogs reposting one PR newswire are one confirmation, not five. Cluster sources by syndication before the corroboration boost, and cap it — otherwise press-release amplification games the score. On conflicting facts, structured beats web (SEC > Crunchbase > press > blog); the earliest credible date wins the event timestamp.

One shared event-key function serves both the pull Resolve + dedup step and the push system’s dedup gate, so the global cache holds exactly one canonical row per event — the two pipelines converge instead of writing rival rows.
06

The enrichment waterfall

Signals decide who matters; enrichment fills in the durable company facts behind the lead. On a company-facts miss the run cascades down the tiers in fixed cost order — the model only decides escalate-or-stop per tier against a completeness target, never reorders them. Every tier has the same exit: the moment the target is met, the fields land and the cascade stops. Click any tier to inspect it.

miss · escalatemiss · escalatemiss · escalatecache hit · skip all tierstarget met · stoptarget met · stopverified fields0Intelligence DBcache · free1CheapExa · $2Expensivestructured · $$3Web fallbackweb_search · last resortFacts completewrite-back

Intelligence DB · cache · free

cache read · $0
deterministic

Global company-facts cache keyed by domain, TTL ~90d. A hit returns firmographics instantly and skips every paid tier — the “enrich faster next time” payoff. Signals are never read here; they’re re-discovered live every run.

domaincached firmographics
  • cost free · rep —
  • TTL ~90d
  • hit = zero provider spend
RULEStructured beats web

On a field conflict the higher-reputation source wins. A structured-provider value is never overwritten by a scraped one; web values must be cited and verified before they’re stored at all.

!Web search is grounded, never trusted

Web search hallucinates ~30% even on strong models and fabricates 3–13% of cited URLs. So it sits last, every field it fills must appear in its cited page (enrichment_verify.py), and anything unverified is stored at low reputation — which caps its own contribution to the score.

07

Composite scoring

The lead score is computed in code — no override-rule prompt, no LLM-emitted number. Compose a few signals below and watch the score move; each weight here is a tunable constant in the real config.

9/10
reach out now
fit × Σ( type × strength × source × recency )
Σ raw 2.56× fit 1.30= adj 3.33
presets
ICP / offer fit
1.00
0.90
0.66

Curve is illustrative of the deterministic formula — exact constants are tuned at build from eval data. The point: the number is reproducible code, not model judgement.

The source term is corroboration-aware. The sandbox above shows one observation per row — the single-source floor. In production a canonical signal carries every source that reported it (see section 05), so the term is confidence = f(best source reputation, count of independent sources, agreement): a funding round in SEC + Crunchbase + two independent outlets outranks one unverified blog. The boost counts independent sources only — five blogs reposting one newswire are one confirmation, not five.
08

The intelligence database

A global canonical store keyed by domain, read first in the waterfall and written back after every run. Public data (firmographics, signals) is shared across orgs — enrich once, reuse everywhere — while subjective judgement (score, fit, notes) stays in a per-org overlay behind RLS.

intelligence_companiesglobal

domain PK · firmographics · per-tier verified_at (TTL: firmographics ~90d).

intelligence_signalsglobal

domain FK · type · event_date · source · source_reputation · confidence · first_seen (TTL ~7–14d).

intelligence_aliasesglobal

raw_name → domain. Instant re-match without re-resolving.

org_company_overlayper-org

org_id + domain · lead_score · fit · notes. RLS-scoped — judgement never leaves the tenant.

workflow_run_companiesunchanged

The per-run output contract. Frontend + CRM sync untouched by the rebuild.

TTL on the company-facts cache is what makes a cache hit cheap: firmographics rarely change (~90d), so a known company skips the enrichment waterfall entirely. Signals are never read from cache on a search — they’re perishable and re-discovered live every run; their ~7–14d TTL only governs how long a written-back signal stays useful to the shared store (the monitoring product reads it; search does not). Scores are never cached (they depend on the searching org’s ICP).
09

Clay-style progressive save-back

The run already owns an SSE channel for step progress; save-back reuses it. As cells resolve, the orchestrator emits tiny per-field frames on that same stream and the frontend applies each one to a live table — no polling, no re-fetch. Every frame is keyed by run_company_id (the workflow_run_companies row), so the table is just a reactive map the frames patch in place.

Skeleton first

Rows appear immediately

At resolve/dedup, skeleton workflow_run_companies rows are inserted in a loading state — the table fills with placeholders before any enrichment finishes, so the user sees motion instantly.

Patch frames

Cells fill as they resolve

Each enriched field streams as a tiny per-field patch frame on the existing run SSE — the same plumbing that already carries step progress. Token-efficient: a frame is a field delta, not a re-serialized company.

Run-SSE frame schema

What crosses the stream

run.progress{ step, pct }Advance the step rail / run status — the channel's original job.
row.skeleton{ rcid, domain }Insert a placeholder row at resolve/dedup, loading state.
cell.patch{ rcid, field, value, source }Set one cell in place — an idempotent field write.
score.patch{ rcid, lead_score }Fill the score cell once enrichment + composite score land.
run.done{ run_id }Finalize — table reconciles against the durable rows.
Applying a frame: the table is a reactive Map<run_company_id, row>. A cell.patch sets row[field] = value in place, so frames are order-independent and a duplicate on replay is a harmless overwrite — no per-frame reconciliation logic. On reconnect, EventSource replays the run's backlog from Last-Event-ID; on run.done the table reads the durable workflow_run_companies rows, so a watched fill and a cold load converge to the same state. Frames are ephemeral and die with the run — fine, because the database write is the durable truth.
10

Bounded autonomy

The two agentic nodes run under hard ceilings. Discovery stops at the first limit; enrichment’s order is fixed and the model only decides whether to escalate a tier.

Discovery loop
max_tool_calls~15Bounds the agentic fan-out; stop at the cap
max_sourcesconfigCaps registry breadth per run
wall_clockconfigHard time ceiling
token_budgetconfigPer-run $ / token ceiling
Enrichment waterfall
orderfixedcache → cheap → expensive → web
decisionstop / escalateOnly call the model makes per tier
completeness_targetrequired fieldsdomain, industry, size, ≥1 dated signal
cache-hit short-circuitzero tiersA company already in the intelligence DB skips the waterfall — discovery still runs
11

Known failure modes

Enrich

Web-search hallucination

Web search hallucinates ~30% even on strong models; 3–13% of cited URLs are fabricated. Mitigation: web is the last tier only, every field must appear in its cited page (enrichment_verify.py), and unverified values are stored low-reputation — so they self-limit their score contribution.

Dedup

Same-named distinct companies

Two different companies can share a name. Mitigation: domain is the canonical key, resolved before dedup; fuzzy-name clustering is a fallback only when no domain resolves.

Score

Cold-start with no profile

A new org has no playbook to shape fit. Mitigation: fit_multiplier defaults to ~1.0 and ranks on signal strength × recency + a generic marketing-buyer heuristic; the UI nudges playbook creation.

Loop

Runaway agentic spend

A tool loop with no ceiling can burn budget on a hard brief. Mitigation: hard caps on tool calls / sources / wall-clock / tokens; the loop returns partial results rather than overrunning.

Cache

Cold global cache

On day one every run pays full enrichment. Mitigation: accepted — the cache warms across orgs, so hit-rate and cost improve with use.

Trust

Single-source low-rep signals

One unverified source could pollute the global store. Mitigation: stored flagged with source attribution; reputation weight caps downstream impact. Corroboration-before-share is a v2 tightening.

12

Design decisions

Runtime

Fresh build on the Agent SDK

Not a port of the Agno DAG — a new deterministic skeleton with two bounded agentic nodes. Tools run host-side (Exa / Explorium / Parallel / Hunter keys never enter a sandbox).

Scoring

Deterministic, not LLM-emitted

The score is a code formula over extracted signals. Testable, tunable per-weight, no prompt drift — retiring the ~15 override rules of the old auditor prompt.

Autonomy

Bounded where it pays

The model has autonomy in two places only — which sources to query, and when to stop enriching — each hard-capped. Everything else is deterministic code.

Data

Global cache, private judgement

Public firmographics + signals are shared globally (enrich once, reuse everywhere); lead_score / fit / notes stay in the per-org overlay behind RLS.

Deferred to build time (not blocking the design): eval harness (golden-run capture off the live path → shadow-compare → gate on score-correlation + recall); per-node model tiering (opus-4-8 for discovery/strategist reasoning, haiku for extract/classify); corroboration-before-share for global writes; and regenerating the pipeline diagrams to match this spec.
AgencyCore · Company Signals design speccompany signal discovery · Claude Agent SDK