TECHNICAL DEEP-DIVE // ARCHITECTURE
How NOVA actually works
A medallion pipeline (Bronze → Silver → Gold) on ClickHouse, orchestrated in batch by Dagster and dbt, running alongside a second, always-on daemon that evaluates and delivers alerts directly — the two paths share one ingestion process but nothing else. Below: every component, the exact hops data takes, and the production notes worth knowing before touching any of it.
Components
A uv workspace monorepo: one Python environment, three installable packages, plus a dbt project and a Terraform config that sit outside it.
| Component | Built with | Role |
|---|---|---|
| pumpfun_common | pydantic v2, clickhouse-connect | Shared models, Bronze table/column name constants (single source of truth the daemon and dbt sources.yml both read from), ClickHouse client factory, DDL bootstrap runner. |
| ingestion | asyncio, httpx, websockets, tenacity, aiolimiter | The always-on daemon. Holds the PumpPortal and Helius connections, decodes trades, writes Bronze, emits a heartbeat, and runs the real-time AlertEvaluator in the same process. |
| orchestration | Dagster, dagster-dbt | Wraps the dbt project as @dbt_assets, represents Bronze as observable source assets, runs an ingestion_daemon_is_alive asset check off the heartbeat table, schedules the dbt build every 10 minutes. |
| dbt project | dbt-clickhouse | Silver staging models and Gold marts as plain SQL, plus schema tests that surface as Dagster asset checks automatically. |
| bot | aiogram | A separate package for the interactive /check and /top_insiders commands, reading the batch Gold layer — fully independent of the alert-delivery path. |
| ClickHouse | ClickHouse server + dbt-clickhouse adapter | Bronze (files + a lookup index + a heartbeat table), Silver and Gold as real materialized tables in their own databases. |
| Terraform | azurerm ~>4.0 | Provisions Azure Container Apps, ACR, Key Vault and Azure Files for the cloud deployment; a separate bootstrap config manages remote state. |
From a websocket to a Telegram message
One daemon process, two live upstream connections, three things written per event, and a fork into a real-time path and a batch path.
Launches arrive over PumpPortal's free subscribeNewToken websocket — no API key, no funded wallet, unlimited. Trades are more involved: PumpPortal's trade subscriptions turned out to require a funded wallet and are metered per event, so NOVA instead opens a Helius RPC websocket and calls logsSubscribe filtered to the pump.fun bonding-curve program (6EF8rr…F6P). That only returns a bare signature, so a follow-up getTransaction resolves the actual trade — often not on the first try, since Helius nodes can return result: null for several seconds after the notification fires, so the fetch retries with backoff. The decode itself doesn't touch Anchor instructions at all: diffing pre/postTokenBalances and pre/postBalances on the resolved transaction is enough to recover trader, side, and amounts.
Every decoded event — launch or trade — is written three ways: appended as one JSON line to a rotating .jsonl file (Bronze's actual storage), indexed as a lightweight row in a real MergeTree table for fast lookups, and reflected in a heartbeat row roughly every 10 seconds so a quiet market can be told apart from a dead daemon. From that same event, two independent things happen next.
Bronze, Silver, Gold
A medallion layout, but Bronze is deliberately not a ClickHouse table — it's a query layer over files ClickHouse never writes to.
| Layer | What it is | Format | Written / refreshed |
|---|---|---|---|
| Bronze | Raw events exactly as received — one line per launch or trade, plus a heartbeat | Rotating .jsonl files (raw_events view via ClickHouse's file() engine); a MergeTree index table; a typed heartbeat table | Continuously, by the daemon |
| Silver | Cleaned, typed staging models — stg_token_launches, stg_trades | ClickHouse tables, dbt-managed | Every dbt build (~10 min, Dagster-scheduled) |
| Gold | Rule-based marts — dim_token_risk_score, dim_insider_wallets | ClickHouse tables, dbt-managed | Every dbt build (~10 min, Dagster-scheduled) |
Trade payloads keep both a decoded block (trader, side, amounts — the balance-diff output) and the untouched original Helius response under raw, so nothing is lost to the reshaping. The index table exists only because file/URL-engine sources have no data-skipping stats of their own — every query against raw_events otherwise re-scans every matching file. Silver and Gold don't read the index; they query raw_events directly and extract fields with JSONExtract*.
Local Docker Compose → Azure Container Apps
Same three images, two very different topologies for how they reach ClickHouse's storage.
The container apps reference images that have to already exist in the ACR the same Terraform config creates, so apply runs in two passes: infra-only resources first (-target=…), then make acr-push, then the rest. ACR has admin_enabled = false — every app authenticates via a system-assigned managed identity and an AcrPull role assignment, not shared credentials. The Bronze DDL bootstrap is a manual-trigger container app job (az containerapp job start), not scheduled — idempotent, but nothing needs it to re-run on its own.
What broke, and what it taught
Six things that cost real debugging time, kept here so nobody re-discovers them.
A ClickHouse 429 isn't valid JSON
Helius rate-limit responses aren't JSON, and json.JSONDecodeError isn't an httpx.HTTPError subclass — it was escaping uncaught from the fire-and-forget trade-fetch task, silently dropping that signature.
Fix: caught explicitly and treated as a retryable miss.
The ClickHouse image chowns your bind mount
The official image's entrypoint chowns user_files_path to its internal clickhouse user on every start when running as root — fighting a host daemon that needs the same directory writable.
Fix: run the container as the host UID so the root-only chown step never fires.
ClickHouse's analyzer drops implicit aliases
An implicit t.mint → mint alias didn't propagate to an outer scope when the CTE producing it itself contained multiple joins, throwing UNKNOWN_IDENTIFIER.
Fix: make the alias explicit — t.mint as mint — in any CTE-with-joins.
SIGTERM doesn't always land
A plain kill on a nohup'd daemon left it running even after a wait and a repeat SIGTERM.
Fix: escalate to SIGKILL and verify with ps -p <pid> — never assume a kill worked.
The free public Solana RPC's log filter is a no-op
Subscribing to logsSubscribe with mentions set to the pump.fun program on api.mainnet-beta.solana.com returned ~35,000 messages in 45 seconds — zero of which referenced that program.
Fix: moved trade streaming to Helius, whose free tier honors the filter.
PumpPortal trades aren't actually free
subscribeTokenTrade / subscribeAccountTrade need an API key and a wallet funded with SOL, metered per event — only the launch feed is free.
Fix: kept PumpPortal for launches only; trades go through Helius instead.