FrankBuild governed data pipelines without rebuilding the data platform.

Low-code EL/T for lakehouse teams -- source discovery, durable syncs, AI-assisted transforms, pipeline DAGs, ontology publication, and a CLI/SDK surface that works for humans, CI, and agent runtimes.

Quickstart

Concepts

Architecture

Sources land cleanly in Bronze

Register databases, SaaS systems, APIs, files, streams, or warehouses through source patterns. Frank discovers streams, configures sync modes, wraps records in a data envelope, and writes tenant-isolated Iceberg tables.

Configure sources

Transforms are independent data products

Build SQL, dbt, and Python-runner transforms that consume one or many Bronze or transform outputs. Source syncs and transform lifecycles are separate, so EL can fail without invalidating downstream logic.

Build transforms

Pipelines compose real DAGs

Chain transforms into versioned pipeline DAGs with topological validation, immutable snapshots, sandbox runs, activation, pause/resume, and terminal step classification.

Compose pipelines

AI helps where humans lose time

Martha-backed workflows suggest target schemas, field mappings, pattern parameters, SQL reviews, bespoke transform code, CI fixes, and full pipeline compositions.

Use AI assistance

Ontology publication is first-class

Map curated Iceberg tables to ontology entity types, sync rows into ontology-core-v2, track sync history, and expose backing datasets for semantic applications.

Publish to ontology

Pattern catalog, not hardcoded connectors

Source patterns and transform patterns live as declarative config. Add a source, transform, geospatial operation, validation rule, or Python runner without changing the UI.

Pattern reference

API, UI, CLI, and SDK stay aligned

The SvelteKit UI, FastAPI routes, frankctl, and frank-sdk all speak the same primitives: sources, streams, transforms, artifacts, runs, pipelines, datasets, schedules, and patterns.

CLI guide

Operable by default

Temporal handles source and transform workflows; Dagster executes materializations; Loki stores logs; OpenTelemetry traces API and worker paths; run records keep the UI fast.

Operations guide

Why Frank

Most teams do not need another pile of scripts around a warehouse. They need a way to turn messy source systems into trustworthy, explainable, reusable data products.

Frank builds the repeated platform work once: source setup, stream discovery, lakehouse naming, data envelope metadata, transform hydration, run tracking, lineage, schedules, ontology publication, and the AI help that removes blank-page work from schema and mapping tasks.

Frank handles the platform layer:

You do not build	You do build
Source discovery, sync workers, cursor state, Iceberg writes	Which source systems to connect
Pattern registry, dynamic config forms, connector adapters	Source-specific credentials and stream choices
Transform hydration, artifacts, run lifecycle, logs	The business shape of your transform
DAG validation, versioning, sandbox execution	The steps and dependencies in a pipeline
FIWARE schema lookup, ontology proxy, backing dataset sync	Which entities your data products should publish
AI orchestration, prompt routing, trace IDs	Human review and promotion decisions
CLI/API contracts, tenant scoping, service auth	How your team automates rollout

Concepts defines the primitives. Quickstart gets the local stack, UI, API, and CLI running.