Company Intelligence Database
The shared company intelligence layer: one global, org-free record per real-world company, written by sonar runs, scrapers and agents, and read first by every enrichment step. A fresh hit costs nothing. A miss enriches once, writes back with provenance, and every tenant benefits. Tenant CRM state never crosses the seam.
Two layers, one seam
tl;dr Today every org pays full enrichment price for every company, even one another org enriched yesterday: crm_companies_v2 and workflow_run_companies are org-scoped, so knowledge never crosses tenants. The intel layer fixes that with a clean split: a global intelligence layer that improves with network effects, and the tenant-private CRM that never leaks into it.
Company intelligence
Canonical company profiles with per-tier freshness, written once and read by every tenant's runs - by enrichment, scrapers, cron jobs and agents. Raw payload and cost lineage live in the shared intel_sources table.
CRM workflow
Ownership, lists, stages, notes, tasks, outreach and run snapshots. Each org keeps its own copy of a company; intel only pre-fills the enrichment fields.
System design
before / after Two interactive entity diagrams. Production today is the org-scoped world: crm_companies_v2 and workflow_run_companies live per tenant, so every org re-enriches the same company and pays the provider chain again. The intel layer design adds a global, org-free intel_* layer in front of the unchanged tenant CRM. Toggle the two, then click any table to trace its relationships and columns.
The new design: a global, org-free intel_companies in front of the unchanged tenant CRM, written once and read by everyone. Hover or click a table.
The new intel_companies profile deliberately mirrors the enrichment fields crm_companies_v2 already stores - identity, firmographics, funding. The structural difference is the boundary: public facts move into a shared layer that has no organization_id, while tenant CRM state stays exactly where it is. Same columns, one new place for the shared half to live.
Data flow
interactiveHow a single request moves through the new design. Pick a path - a cache hit, a miss that writes back, or the tenant pre-fill seam - and the diagram lights the nodes and edges it touches, in order. Click any node for detail.
Fresh hit: the run asks the facade, the lookup finds a fresh profile in intel_companies, and it is served straight into the result and the tenant CRM row. Zero provider calls, $0 spend.
Click any node for detail.The write-back path is what turns a per-org cost into a one-time cost: the first tenant to touch a company runs the provider chain, the result lands in intel_* with provenance, and every later run by any tenant follows the cache-hit path at $0. The pre-fill seam is one-way by construction - nothing tenant-derived ever travels back up.
Read-through cache
interactiveThe core mechanic: enrichment asks the intel layer first, providers run only on a miss, and every miss makes the cache smarter. Pick a scenario and watch the lookup path.
Today the Exa → Parallel → GPT website chain and the Hunter → Explorium firmographic chain run per org, per run. With the cache in front, the network gets cheaper as it grows: the more companies any tenant touches, the higher everyone's hit rate.
Privacy boundary
seam The seam is enforced twice: by policy (only externally observable, public facts may enter the shared layer) and by mechanism (client roles get RLS deny-all on intel_*; the ac-python-api service role is the only door).
intel_companiesintel_sourcesProfiles and their public facts flow down into sonar results and tenant CRM copies.
Provider and scraper output discovered during a run flows up with provenance.
Tenant workflow state, lead scores and who-saved-what never enter the shared layer.
crm_companies_v2workflow_run_companiesnotes · tasks · outreach- Firmographics: industry, size band, locations, founded year
- Funding rounds, IPO status, acquisitions
- Website content, positioning, tech stack
- Hiring activity and public job posts
- News, ads and other externally observable events
- CRM stage, owner, notes, tasks, outreach history
- Whether any tenant saved, rejected or shortlisted a company
- Per-tenant ICP scores, lead scores, signal feed state
- Anything derived from tenant activity, even aggregated
Caching paid provider output cross-tenant must be cleared against each provider's terms (Hunter, Explorium in particular). Scraped and LLM-derived facts are unambiguous; provider payloads need the review noted in the rollout section.
Data Schema
model One company-specific table: the flat intel_companies profile, the fast read path for every tenant. Per-tier freshness stamps live on the row, so there is no field-level facts table to assemble on read. Raw provider payload and cost lineage sit in the shared intel_sources table, designed on the signals page. Click to open it.
intel_companiesCanonical flat profile - one row per company, the fast read pathdomaintext · uniquenormalized, primary identity keylinkedin_urltextfallback identity for domainless companiesname · website · description · logo_urltextindustry · sub_industrytextemployee_count_band · funding_total_numeric · ipo_statusnumeric / textmirrors crm_companies_v2 enrichment fieldstech_stacktext[]extrajsonblong-tail provider fields until usage proves they deserve columnssource_idsuuid[]fk into the shared intel_sources lineage table (designed on the signals page)fetched_hot_at · fetched_warm_at · fetched_cold_attimestamptzper-tier freshness stamps drive the TTL check - no facts table neededfirst_seen_at · last_enriched_attimestamptzAcme is enriched once, then read by everyone. The schema made tangible. The first tenant's sonar run misses, so the provider chain runs once and two payloads land in the shared intel_sources lineage. The flat intel_companies row is then upserted: its source_ids cite those fetches, and the per-tier freshness stamps are set so every later lookup checks the clock, not the providers. Every other tenant reads this one row for $0. Follow the keys down the cards: src_h1 / src_x2 the fetches, acme.com the row they build.
intel_sourcesmiss-time fetch - Hunter + Explorium firmographics, run once for all tenantsintel_sourcessame miss - Exa to GPT website resolution chainintel_companiesthe write-back - one flat row, source_ids cite the fetches, per-tier stamps drive future TTLFirst-class columns match what the CRM copy already stores (industry, size band, funding, ipo_status), so pre-filling a tenant row is a straight field map. Long-tail provider data stays in extra until product usage earns it a column - no per-field facts table, and provenance reuses the shared intel_sources lineage table rather than redefining one here.
Decision register
decidedThe six decisions the earlier workspace left open, now closed. Each tab shows the call and why it went that way.
What identifies a company?
decidedNormalized domain is the canonical key
Exact normalized domain (lowercase, strip www) auto
Canonical LinkedIn company URL when no domain exists auto
Fuzzy name + country suggest only
Domain matches the existing (organization_id, website) uniqueness pattern and is cheap to normalize at write time. In a shared layer a bad merge is no longer one org's problem, so fuzzy name + country never merges automatically.
intel_sources lineage table and the signal model are designed on the Signals intelligence database. The paired people-side design lives at People intelligence database.