EKOS compiles GitHub, Pentaho, SQL, and more into an evidence-backed ledger — but
until now the only way to read it was through an agent or a query. ekos docs
generate renders that same compiled knowledge as Markdown, self-contained HTML, and
Mermaid diagrams: pure rendering, zero LLM calls, zero cost, with an opt-in citation-validated
prose tier on top. Every example on this page is real output from a real recovered SQL schema —
nothing staged.
ekos docs generate$ ekos docs generate Documentation generated. Format: md Objects rendered: 22 Diagrams rendered: 1 Output: docs-generated
13 real recovered Table objects, 2 real File
objects, and 7 real TransformNode objects from a
CREATE VIEW ... JOIN — every kind gets a page except
Column, which stays embedded in its parent table's properties. One
ER diagram. No API key required, no network call made.
$ git clone https://github.com/joseph-higaki/\ etl_adventureworks_sales_purchases_datamart $ ls dim_customer.ktr dim_vendor.ktr dim_date.ktr fact_purchase.ktr dim_product.ktr fact_sales.ktr dim_sales_person.ktr DimensionsJob.kjb dim_sales_territory.ktr FactsJob.kjb DB Scripts/ readme.md
8 real .ktr transforms + 2 real .kjb
jobs — the same real repo used elsewhere in this project's own recovery testing (devlog_29,
devlog_31), not a synthetic fixture built to look good. ekos build &&
recover && resolve && compile && commit, then
ekos docs generate — same command, no Pentaho-specific flag.
Recover complete. SQL files analysed: 5 Git commits analysed: 1 Local documents analysed: 4 Pentaho jobs analysed: 10 Transformation IR nodes (Pentaho): 86 total, 49% mapped (non-Unmapped) Transformation IR nodes (SQL): 12 total, 100% mapped (non-Unmapped) Documentation generated. Objects rendered: 198 Output: docs-generated
198 real objects: 21 Files, 86
TransformNodes across 10 Pentaho jobs, a
Person object recovered from real git commit authorship, and
Section objects from the repo's own PDF slide deck — the same
file/document/person recovery this deck's other examples show, all from one real project, one
command.
# fact_sales.ktr:10 (TransformNode) ## Properties | `join_kind` | Left | | `node_type` | Join | ## Relationships ### FeedsInto - → fact_sales.ktr:16 - ← fact_sales.ktr:0 - ← fact_sales.ktr:14 ## Evidence - Left JOIN ON [] (confidence: 1.00)
```mermaid
graph TD
n_0["fact_sales.ktr:0"]
n_14["fact_sales.ktr:14"]
n_10["fact_sales.ktr:10"]
n_16["fact_sales.ktr:16"]
n_0 -->|FeedsInto| n_10
n_14 -->|FeedsInto| n_10
n_10 -->|FeedsInto| n_16
A real Kettle Join Rows step from a real fact-table
pipeline — two upstream steps feeding one join, one downstream step consuming it. No hand-built
example: this is step 10 of the real fact_sales.ktr, exactly as
recovered.
# dim_customer.ktr:1 (TransformNode) ## Properties | `node_type` | Unmapped | | `reason` | unrecognized step type: Sequence | | `raw` | <step><name>Create surrogate key</name> <type>Sequence</type>... </step> ## Relationships ### FeedsInto - → dim_customer.ktr:4 - ← dim_customer.ktr:5
49% of this repo's Pentaho steps mapped onto the Transformation IR; the rest —
like this real Sequence (surrogate-key generator) step — become
Unmapped with the exact reason and the full raw XML preserved as
evidence, still wired into the same diagram with its real upstream/downstream edges. "Unmapped
is a citizen, not a failure" — the same rule this whole project runs on, visible in the
generated docs themselves, not just the code.
# "Order Details" (Table) ## Relationships ### ForeignKey - → Orders (`e6969c91-...`) — evidence: "order details".OrderID → orders.OrderID - → Products (`72719579-...`) — evidence: "order details".ProductID → products.OrderID ## Evidence - `123db216-...` — CREATE TABLE "Order Details" (confidence: 1.00) - `6c11b1d1-...` — "order details".OrderID → orders.OrderID (confidence: 1.00)
Relationship targets resolve to real names, not raw ids — a gap only found by rendering this exact file against real data (devlog_34) and fixed the same session.
```mermaid
erDiagram
"Order Details" }o--|| "Orders" : references
"Order Details" }o--|| "Products" : references
"Products" }o--|| "Categories" : references
"Territories" }o--|| "Region" : references
### FeedsInto ← northwind.sql#13:5 ```mermaid graph TD n_5["northwind.sql#13:5"] n_5 -->|FeedsInto| n_6["northwind.sql#13:6"]
The transformation DAG needed no new code — a real
CREATE VIEW ... JOIN ... JOIN compiled into
Custom("TransformNode") objects linked by
Custom("FeedsInto") relationships, the exact shape the generic
dependency-graph renderer already draws for any object. Three planned diagram families
collapsed into two renderers once real data showed they were the same mechanism.
--format html<h1>"Order Details" <span class="kind">(Table)</span></h1>
<h3>ForeignKey</h3>
<ul>
<li>→ Orders <code>e6969c91-...</code>
— evidence: "order details".OrderID → orders.OrderID</li>
</ul>
Embedded CSS, not a build-time include of this repo's own site theme —
ekos docs generate runs in arbitrary user workspaces that don't have
this repo's files available. Mermaid is shown as source in a <pre>
block, not live-rendered: rendering it would need bundling or CDN-loading
mermaid.js, which conflicts with staying fully offline. Stated as a
limit, not fixed silently. The hosted examples linked on this page add mermaid.js
from a CDN on top of the unmodified generated HTML, purely for this demo — the generator's own
output never does that by default.
# Generated Documentation ## Diagrams - [Entity-Relationship Diagram](er-diagram.md) ## File (2) - [northwind.sql](file-northwind-sql.md) ## Table (13) - ["Order Details"](table-order-details.md) - [Categories](table-categories.md) … 11 more ## TransformNode (7) - [northwind.sql#13:0](transformnode-northwind-sql-13-0.md) … 6 more
--prose — real local LLM, not a mockekos ask's citation validation exactly. Never a new pipeline.$ ekos docs generate --prose Prose generation requested for 22 page(s). Estimated input tokens: ~7901 Proceed with these LLM call(s)? [y/N]: y
## Overview The northwind.sql file contains a SQL query that joins three tables: Orders, Customers, and Order Details... _Cited evidence: `6c66cfe7-...`, `97228874-...`, `2e516286-...`, `af5279d1-...`_
Real end-to-end run against local llama3, not a mock:
22/22 objects got a real Overview, 16/22 with real citation-validated evidence ids. The other 6
degraded honestly — full answer kept, empty citation list — the same "answer is never discarded"
contract ekos ask already had. Across every one of the 22 real calls,
zero fabricated citations survived: a bogus id mixed into a mock response in testing never made
it into cited_evidence, and the real run only ever cited ids that
actually exist in the ledger.
<pre>, not rendered — the offline-first tradeoff.// full sentence as the ask() question: "Write an overview of X: what it is..." // buries the name deep enough that // retrieval matches nothing → every // citation gets dropped, even valid ones // fixed: just the object's name ai.ask(&model.name)
--layout curated — RFC 0037/0042, added since this deck's first slides$ ekos docs generate --layout curated --output doc
Curated documentation generated.
Objects considered: 4267
Files: README.md, Architecture.md, API.md,
SequenceDiagrams.md
Entity detail pages written: 1855
Output: doc
Architecture.md now includes a real crate/workspace dependency
graph parsed from every Cargo.toml (not guessed), real CI/CD
pipelines parsed from .github/workflows/*.yml, and
API.md links every real function/struct/enum/trait
(RustSymbol/PythonSymbol) to its own
detail page — 1,855 of them for this repo alone, nested under
entities/<kind>/<shard>/ so the page count never blows
past GitHub's per-directory listing cap. Same zero-LLM, zero-cost rendering this whole deck
already demonstrated — just a different, more opinionated shape.
# compile once $ ekos build && ekos recover && ekos resolve && ekos compile && ekos commit # deterministic, zero LLM, zero cost $ ekos docs generate --format html $ ekos docs generate --layout curated --output doc # opt-in, citation-validated overview per page $ ekos docs generate --prose