EKOS — Enterprise Knowledge Operating System

Pentaho already compiled.
Now compile it into dbt.

EKOS recovers Pentaho ETL logic into the Transformation IR (RFC 0027) — the same compiled graph ekos docs generate already renders as documentation. ekos dbt generate renders that same graph a second way: real dbt SQL models, ref()/source()-chained via the compiled data-flow edges. Every example on this page is real output from a real cloned Pentaho repo, unedited — including the five real bugs it found, and fixed, along the way.

Pentaho .ktr/.kjb Transformation IR dbt SQL models + schema.yml
§ 01 / the command
One command, the ledger already has everything it needs

No new extraction. Just a second rendering of what's already compiled.

terminal — real run, real Pentaho repo
$ ekos dbt generate
dbt models generated.
  Models rendered: 98
  Source tables: 23
  Output: dbt-generated

98 real dbt .sql models from 10 real Pentaho jobs — one file per compiled TransformNode, ref()-chained via the same real FeedsInto edges the documentation deck's diagrams already draw from. Zero LLM calls, zero new analyzer: this reads the exact graph ekos docs generate reads.

§ 02 / source → dbt source()
dim_customer.ktr, step 6 — real, unedited

A real TableInput step becomes a real dbt source.

source — Pentaho .ktr XML
<step>
  <name>Sales Person</name>
  <type>TableInput</type>
  <connection>AdventureWorks</connection>
  <sql>SELECT
  BusinessEntityID
, TerritoryID
FROM Sales.SalesPerson
</sql>
target — dbt-generated/dim_customer_ktr_6.sql
select
    BusinessEntityID,
    TerritoryID
from {{ source('pentaho', 'Sales.SalesPerson') }}

The FROM clause resolves into a real dbt source() reference, and the real declared columns — read from the step's structured <row-meta>/<value-meta>/<name> metadata, not parsed out of the free-text SQL — become a real explicit column list instead of select *.

Open the real file: source-dim_customer_ktr_6.sql
§ 03 / a real gap, found — and fixed
fact_sales.ktr, step 10 — real, unedited

StreamLookup and MergeJoin have their own key XML shapes. Now both are read correctly.

source — Pentaho .ktr XML (StreamLookup)
<step>
  <name>Sales Territory Lookup</name>
  <type>StreamLookup</type>
  <lookup>
    <key>
      <name>TerritoryID</name>
      <field>territory_id</field>
    </key>
  </lookup>
target — dbt-generated/fact_sales_ktr_10.sql
-- NOTE: l/r assignment is positional...
select *
from {{ ref('fact_sales_ktr_0') }} as l
left join {{ ref('fact_sales_ktr_14') }} as r
    on territory_id = TerritoryID -- TODO: verify column
       qualification, source dialect: Pentaho

Real finding from this session's Phase 2 testing, now fixed: every real MergeJoin/StreamLookup step in this repo first compiled with empty keys — each uses its own real key XML shape (<lookup><key><name>/<field> for StreamLookup; <keys_1>/<keys_2> for MergeJoin), neither matching DatabaseJoin's <keys><key><value1>/<value2> shape the analyzer originally only read. Both shapes are parsed correctly now — real keys, real join kind, real topology, for every real join in this repo.

Open the real files: StreamLookup example · MergeJoin example
§ 04 / honest, not hidden
dim_customer.ktr, step 1 — real, unedited

A step EKOS can't translate still gets a real model file, still wired into the DAG.

dbt-generated/dim_customer_ktr_1.sql — real, unedited
-- Unmapped: unrecognized step type: Sequence
-- Raw source:
-- <step><name>Create surrogate key</name>
--         <type>Sequence</type>... </step>
select * from {{ ref('dim_customer_ktr_5') }} -- passthrough stub, not translated

A real Kettle Sequence (surrogate-key generator) step — no structured semantics to render, so it becomes an honest passthrough: the raw XML and reason preserved as a comment, still ref()-chained to its real upstream, so the pipeline's shape stays connected end to end even where content isn't fully translated. "Unmapped is a citizen, not a failure" — the same rule the documentation generator runs on, applied to a second output format.

Open the real file: unmapped-dim_customer_ktr_1.sql
§ 05 / sinks and schema.yml
Real output, real project structure

Every model file resolves to a real ref(). A real schema.yml ties it together.

dbt-generated/dim_date_ktr_10.sql — real sink
select * from {{ ref('dim_date_ktr_3') }}
dbt-generated/schema.yml — real, excerpt
version: 2

sources:
  - name: pentaho
    tables:
      - name: "Sales.SalesPerson"
      - name: "Sales.Customer"
      - name: "fact_sales"
      ... 20 more, real

models:
  - name: dim_customer_ktr_0
  ... 97 more, real

23 real source tables, 98 real models — every sources: entry traces to a real compiled Source node's object_name, never invented.

§ 06 / what's never guessed
The line this feature refuses to cross

No expression transpiler. Filter and Calculate stay raw, flagged, not translated.

the honest-passthrough contract, real generated shapes
-- Filter:
select * from {{ ref(upstream) }}
where {condition} -- TODO: verify, source dialect: Pentaho

-- Calculate:
select *, {expr} as {output} -- TODO: verify, source dialect: Pentaho
from {{ ref(upstream) }}

Kettle's Filter Rows/Calculator conditions and expressions are raw, un-parsed source-dialect text — RFC 0027 already rejected building a cross-format expression AST as out of scope, since a wrong guess here would be worse than an honest TODO. Every one of these gets inlined exactly as compiled, flagged for manual verification, never silently rewritten.

§ 07 / real bugs, real fixes
What running this against a real repo actually found

Five real bugs. All from real data, none from imagination.

  • Empty Source columnsTableInput steps compiled with columns: [] — the real declared columns were in <row-meta>, never read. Fixed: read from the real structured metadata.
  • Empty StreamLookup keysIts real key shape (<lookup><key><name>/<field>) doesn't match DatabaseJoin's. Fixed: a dedicated extractor for the real shape.
  • Empty MergeJoin keysIts real key shape (<keys_1>/<keys_2>, two separate lists) doesn't match DatabaseJoin's either. Fixed: positional pairing of the two real lists.
  • Double-qualificationSQL-sourced join keys already carry their own table alias; an early version's invented l./r. prefix produced l.o.customer_id — invalid. Fixed: keys pass through unmodified.
  • Confusing no-keys commentThe on true fallback had a "verify column qualification" comment appended with nothing to verify. Fixed: the comment only attaches to real key text.
before → after, on the same real step
select
    *
from {{ source('pentaho',
  'Sales.SalesPerson') }}

-- fixed:
select
    BusinessEntityID,
    TerritoryID
from {{ source('pentaho',
  'Sales.SalesPerson') }}

All five found by running this against a real 198-object Pentaho repo, not by unit tests — the same pattern that found real bugs in the documentation generator (RFC 0035). Three live in the recovery layer (pentaho_analyzer.rs, shared with ekos docs generate), two in this feature's own rendering. Real, messy data keeps finding what hand-built fixtures miss.

§ 08 / the real numbers
This whole deck is one real repo, two real runs
Source repojoseph-higaki/etl_adventureworks_sales_purchases_datamart — a real public GitHub repo
Models rendered98, from 10 real .ktr/.kjb jobs
Source tables23, real object names from compiled Source nodes
Real Pentaho joins with populated keys2 of 2, after the fix — 0 of 2 before it
Real bugs found & fixed this session5 — 3 in shared recovery, 2 in dbt rendering, all from real-data testing
Try it

Point it at anything EKOS has already compiled.

terminal
# compile once
$ ekos build && ekos recover && ekos resolve && ekos compile && ekos commit

# real dbt models, ref()-chained, zero LLM cost
$ ekos dbt generate
EKOS · RFC 0036 (Pentaho → dbt Export) · every example on this page is real output, unedited · github.com/alexeyban/EKOS