Data Layer Overview
The AgencyCore data layer in one map: the org-scoped CRM plane that runs production today, and the global intelligence plane — companies, people, employments, sources, signals and the per-org feed — designed to sit in front of it. Knowledge is written once with provenance and read by every tenant; workflow state never leaves its org. This page shows both worlds and how the core components work together.
Two planes, one seam
tl;dr Everything in the data layer sorts into two planes. The tenant plane is workflow state — sixteen org-scoped crm_ tables isolated by row-level security, where stage, owner, notes and outreach live. The global intel plane is knowledge — externally observable facts about real companies and people, written once with provenance and shared by every tenant. The seam between them is the whole design: public facts flow down, scored signals fan out, and nothing tenant-derived ever flows up.
Knowledge
Canonical company and people profiles, append-only employment edges, the unified signal store and the shared intel_sources lineage. RLS deny-all — the ac-python-api service role is the only door. Acts as a read-through cache in front of every paid provider.
Workflow state
The production CRM: companies, people, deals, activities, communications, lists and pipeline config, one copy per org. Intel only pre-fills enrichment fields; ownership, stages and judgments stay here forever.
System design
before / after Two interactive entity maps. Production today is the org-scoped world: the CRM core, a per-org signals entity, four divergent signal vocabularies and a provider chain that is paid again by every org that touches the same company. The intel design puts the full intelligence layer in front of it — all nine intel_ tables, from the knowledge profiles to the per-org feed. Toggle the two, then click any table to trace its relationships and columns.
Production today: sixteen org-scoped crm_ tables and a first-class signals entity per tenant. Knowledge never crosses orgs — every org re-enriches the same company and the provider chain is paid per org, per run. Hover or click a table.
The intel profiles deliberately mirror the enrichment fields the CRM already stores — identity, firmographics, funding, titles. The structural difference is the boundary: public facts move into a shared layer with no organization_id, signals collapse from four vocabularies into one append-only store, and the tenant CRM keeps exactly the columns it has today.
How the components work together
interactiveThe same machine end to end: runs and agents on the left, the knowledge plane in the middle, activation on the right. Pick a path — a cache hit, a miss that writes back, a user-initiated signal pull or an automated push — and the diagram lights the nodes and edges it touches, in order. Click any node for detail.
Fresh hit: the run asks the facade, the lookup finds a fresh profile in the knowledge plane, and it is served straight into the result and the tenant CRM row. Zero provider calls, $0 spend.
Click any node for detail. The write-back path turns a per-org cost into a one-time cost: the first tenant to touch an entity runs the provider chain, the result lands in the knowledge plane with provenance, and every later run by any tenant follows the cache-hit path at $0. Signals work the same way — one immutable fact, scored per org, with every discovery attributed in intel_signal_ingests.
Freshness & precedence
model Two clocks govern the knowledge plane. Freshness: every profile carries per-tier stamps (fetched_hot/warm/cold_at), so a lookup serves fresh data, stale-serves while queueing an async refresh, or misses into the provider chain. Precedence: when sources disagree, rank wins — provider payloads beat scrapes beat LLM-inferred values.
Signals, hiring activity, news-derived facts
Funding, headcount band, tech stack
Industry, description, positioning, identity keys
- 1Paid provider payloads — Hunter, Explorium — structured, contractual accuracy
- 2First-party scrapes — website, Firecrawl — public and current, less structured
- 3LLM-inferred values — Exa extraction, GPT synthesis — never silently overwrite a higher rank
Signals are never mutated — decay applies when they are read and scored: score = weight × ICP fit × 2^(−days/90). A 90-day half-life, with per-type windows from 7 days (scaled ads) to 90 days (funding). Push paths need score ≥ 4 for the feed, ≥ 7 for the digest; pull paths surface at any score.
Privacy seam
seam The boundary is enforced twice: by policy — only externally observable public facts may enter the shared layer — and by mechanism — client roles get RLS deny-all on intel_*, the ac-python-api service role is the only door, and erasure is first-class with SHA-256 suppression hashes that block re-ingestion.
intel_companiesintel_peopleintel_person_employmentsintel_person_suppressionsintel_sourcesintel_signalsintel_signal_sourcesintel_signal_ingestsFirmographics, titles, work emails pre-fill tenant CRM rows at save time.
One shared signal row materializes per org in intel_org_signal_feed, scored and gated per org.
A run save writes the externally observable subset back up, with intel_sources provenance.
Stage, owner, notes, tasks, outreach history — tenant-derived, never enters the shared layer.
Shortlists, ratings, rejections, per-tenant ICP and lead scores — even aggregated.
crm_companies_v2crm_people_v2crm_deals · activities · comms · listsintel_org_signal_feedFrom here to there
pipelineThe intel design replaces four divergent signal vocabularies with one store, and lands in three phases — each independently shippable. Click a phase for detail.
signals + company_signals / person_signalsintel_signalsCRM entity-page vocabulary — subject_type replaces the two join tablessignal_events + signal_subscriptionsintel_signals + intel_org_signal_feedcompany monitoring — events become store rows, subscriptions become feed scoringperson_signal_eventsintel_signals (subject_type = person)people monitoring joins the same polymorphic storeworkflow_run_companies.sales_signalsintel_signals + intel_signal_ingestsper-run jsonb recompute becomes one shared append-only fact, attributed per findThis page is the map, not the territory. The decided per-entity designs — schema explorers, cache simulators, decision registers — live on the company, people and signals intelligence pages plus the org signal feed; the production CRM tables each have their own page under the CRM section.