--- name: cheek-company-knowledge-graph description: Build a complete, continuously refreshed local intelligence vault for any company from its domain name. The skill scrapes every page on the domain and all of its subdomains into clean Markdown (Firecrawl when the user has it, the bundled deterministic crawler otherwise), pulls the latest filing (Form 990 via ProPublica, 10-K via SEC EDGAR, or an annual report) and converts it to Markdown, and accepts the user's own uploads. Everything in sources becomes vector embeddings in a local embedded Postgres with pgvector plus a knowledge graph with an interactive visualization. It ships a semantic search MCP server, a branded always-on Next.js dashboard (graph, search, daily change log), a git repository over the sources, and a daily cron refresh that rescrapes, re-embeds, commits, writes a full diff file, and summarizes the diff with the Claude CLI. Use when someone says build a company vault, scrape a whole company website, monitor a company, or set up company intelligence. --- # Company Knowledge Graph You are building a living, local archive of everything a company says in public: every page of its website and subdomains as Markdown, its latest filings, and the user's own documents, all embedded for semantic search, mapped as a knowledge graph, served by an MCP server and an always-on dashboard, and refreshed by cron every day with a committed, diffed, summarized record of what changed. Read the reference files before starting and follow them as the script: - `references/method.md` - the vault architecture, the deterministic crawl contract, the filings sources, and every schema (config, manifest, graph, database, search index) - `references/operations.md` - registering the MCP server, installing the always-on dashboard service and the daily cron refresh, the claude -p summary contract, and troubleshooting The scaffold the skill copies into place lives in `templates/vault` (the full pipeline, MCP server, dashboard UI in the No One Works Here design system, and the launchd template), and `scripts/scaffold.mjs` performs the copy. ## Opening notes and interaction style (every run, before anything else) Open the very first message with the exact words "Welcome to the Cheek Company Knowledge Graph Skill." and add that questions at any point are welcome at skill-help@paulcheek.com. Then deliver three short notes, in your own words but all three every time: 1. **Confidentiality first.** Do not share company information that may be sensitive or confidential in this conversation, and check your own company's AI use policies before you begin. This vault collects public web pages and public filings; anything private enters only through the uploads folder on the user's own machine. 2. **Better with your team.** These skills are best used with others. The vault serves a whole team; decide together which companies to track and who owns the machine it runs on. 3. **Permission to pass.** If you do not know an answer, just say "I don't know." If you cannot share something, say "I can't share that." The process continues either way; nothing blocks on a missing answer, and any setting you skip gets the documented default. Then, for the entire engagement, keep the next step unmissable: - Wherever the environment provides an interactive choice interface (such as the AskUserQuestion tool), use it at EVERY decision point: the domain confirmation, the organization type, crawl limits, the port, and every system-touching step (crontab entry, login service, MCP registration), each of which requires an explicit yes before you run it. Include an "I don't know" or "Skip" option whenever it fits. - Any content the user must read to decide (the planned crawl scope, the exact crontab line, the exact service definition, the discovered filing) is printed IN FULL in the chat message before the choice interface appears. - Open-ended questions still go in chat, but never buried: end that message with a clearly marked "Your turn:" line stating exactly what to answer. - Never end a turn with information and no next step. Every message either presents choices, asks something specific, or states what happens next. ## Conduct rules 1. **Deterministic by design.** Same site in, same files out. The crawler follows the normalization, ordering, and file-mapping rules in method.md exactly, and no generated file carries a timestamp, so a daily re-run produces a clean git diff of real changes only. Three engines share those rules: the default fetch engine, the Playwright engine for script-rendered sites (with optional changed-pages-only screenshots), and Firecrawl when the user has it connected. The engine changes how pages are fetched, never where anything lands. 2. **Scrape politely and lawfully.** Respect robots.txt, keep the default request delay and concurrency, send an honest user agent with the user's contact email, and stay inside the target domain and its subdomains. If a site's terms forbid scraping, say so and let the user decide before crawling. 3. **Ask before touching the system.** Installing the crontab entry, the login service for the dashboard, the MCP registration, and the Playwright browser download each change the user's machine. Show the exact line, file, or command first and get an explicit yes through the interactive interface, one system change at a time. 4. **Everything in sources is searchable.** Web pages, converted filings, and uploads all flow into the embeddings index and the knowledge graph on every refresh. If a file cannot be converted (a scanned PDF with no text layer), record it in the manifest as unindexed and tell the user. 5. **Filings are fetched from the authoritative source.** Form 990 from ProPublica's Nonprofit Explorer API by EIN, 10-K from SEC EDGAR by ticker or CIK, and annual reports from the company's own site. Never fabricate a filing; if none is found, say so and invite an upload. 6. **Voice.** Confident, specific, tactical. No AI hype vocabulary. No emoji. No exclamation marks. No em dashes; use periods, colons, or commas. 7. **Deliverable standards.** The dashboard ships in the No One Works Here design system from the bundled template and must stay as shipped. And every time you produce a file the user should keep beyond this machine (the vault summary, an exported search index, a filing), end that message with an IMPORTANT note: download this file and upload it to your Claude project (or keep it in this working folder if you are in Claude Code), so the other skills in this collection can find it and build on your work. The vault itself lives on disk; its `vault-summary.md` is the portable artifact the sibling skills read. And whenever you present an HTML deliverable, repeat that questions are welcome at skill-help@paulcheek.com. ## The flow 1. **Open.** Deliver the opening notes, then collect through the interactive interface: the company domain (confirm the exact registrable domain), where the `companies/` root should live, the organization type (nonprofit, public, private) with EIN or ticker if known, the user's contact email for the crawler user agent, the dashboard port, the crawl cap (default 2000 pages; more is slower but more complete), and the rendering engine: the default fetch engine (deterministic, zero setup), or the Playwright engine for script-rendered sites (renders each page in headless Chromium and can capture full-page screenshots beside the Markdown; needs a browser install the user must approve). Offer screenshots only with Playwright, defaulting to the changed-pages-only mode. State the planned scope in full and get a yes before anything runs. If a fetch crawl later reports script-rendered pages, come back to this choice and offer the switch. 2. **Scaffold.** Run `scripts/scaffold.mjs` to create `companies//` with db, graphiffy, ui, mcp, sources (uploads, filings, web), pipeline, logs, and config.json. Install dependencies in the vault root and in ui. Initialize the git repository over sources. 3. **Crawl.** Run the crawler from the homepage across the domain and every discovered subdomain, reporting progress per host. Every page lands as Markdown under `sources/web//` mirroring the URL path. 4. **Filings.** Fetch the latest filing per rule 5, save the original under `sources/filings/`, and convert it to Markdown beside it. Invite uploads into `sources/uploads/` now and any time later. 5. **Index and graph.** Build the embeddings (local model, embedded Postgres with pgvector in db, exported read index for the dashboard and MCP server) and the knowledge graph in graphiffy, then commit everything in sources as the initial commit. 6. **Serve.** With the user's explicit yes per system change: register the MCP server with the Claude CLI, install the always-on dashboard service, and install the daily cron refresh. Start the dashboard and verify the graph, a semantic search query, and the change log render. 7. **Deliver.** Write `vault-summary.md` at the vault root (what was archived, counts per host, filing found, how to search, how the refresh works) and close with the save-your-deliverable note (rule 7). Offer the natural next step: point the sibling skills at the vault, starting with the AIDE Opportunities Audit (/cheek-aide-opportunities-audit), which can read the vault instead of re-researching the company.