Quickstart

Get from a fresh checkout to a running Frank stack with API, UI, workers, Iceberg, and CLI access.

What you need

• Docker and Docker Compose
• Node.js 18+ for frankctl
• Python 3.11+ if you run backend tools outside containers
• The shared common-infra stack available next to this repo

Frank's Docker Compose file expects services from ../common-infra, including Postgres, MinIO, Iceberg REST, Temporal, Trino, Loki, and related infrastructure.

1. Start shared infrastructure

From the ecosystem root:

cd ../common-infra
docker-compose up -d

Then return to Frank:

cd ../frank-low-code-pipeline

2. Start Frank

make up

This builds and starts:

• FastAPI on http://localhost:8002
• SvelteKit UI on http://localhost:5175
• Source worker for discovery and extraction
• Transform worker for transform execution lifecycle
• Temporal worker for workflow tasks
• API route initialization, database migrations, pattern sync, and SDM seeding

Check status:

make status
curl http://localhost:8002/health

Open:

• UI: http://localhost:5175
• API docs: http://localhost:8002/docs
• Dagster: http://localhost:3000 or the configured Dagster URL

3. Build the TypeScript CLI

cd frank-cli
npm install
npm run build
npm link
frankctl --help

The CLI defaults to http://localhost:8000, so point it at the compose port:

frankctl --api-url http://localhost:8002 status

For repeated local use:

frankctl config set profiles.default.apiUrl http://localhost:8002
frankctl status

In current local dev stacks, some routes require tenant context before full JWT tenant resolution is wired everywhere:

export FRANK_DEV_MODE=true
export FRANKCTL_TENANT_ID="00000000-0000-0000-0000-000000000001"

Use a real tenant UUID if your database has seeded tenants.

4. Discover a source pattern

frankctl patterns list --json

Source patterns include databases, SaaS systems, warehouses, APIs, files, and streams. Transform patterns include filtering, dedupe, joins, aggregation, geospatial operations, validation, conversion, SCD handling, and Python-runner patterns.

5. Create a source

Create a YAML file:

name: local-postgres
pattern_id: postgres
source_config:
  host: host.docker.internal
  port: 5432
  database: app
  username: app_user
  connector_type: postgres
target_config:
  namespace_mode: destination_defined
  table_prefix: raw_

Then create the non-secret Source:

frankctl sources create -f source.yaml

Credential bootstrap depends on the server's explicit SOURCE_CREDENTIAL_MODE.

Current base Compose uses legacy_inline. Submit the pattern's write-only values through the ordinary Source update input; the dedicated credential lifecycle command is unavailable:

frankctl sources update <source-id> -f - < source-credential-update.yaml

The transient update document contains credential_values with only the pattern's declared credential fields (for PostgreSQL, password). Inline mode persists those values in Source.source_config, but Source readback and export redact them and return no credential_ref.

The immutable release uses vault. Only in that mode, stream the exact declared value object into the Vault lifecycle command:

frankctl sources credentials set <source-id> \
  --name local-postgres --values-file - < credential-values.yaml

Vault-mode Source readback contains ordinary configuration and the returned opaque credential_ref, never credential values. Keep either temporary input file untracked and mode 0600, or supply stdin from the approved secret manager. Do not place values in the Source manifest or shell arguments.

After the mode-appropriate bootstrap, discover and inspect streams:

frankctl sources discover <source-id> --timeout 300
frankctl sources streams list <source-id>

Configure streams:

streams:
  - name: customers
    sync_mode: incremental
    cursor_field: updated_at
    write_disposition: merge
    primary_key_path: ["id"]
    is_enabled: true

frankctl sources streams set <source-id> -f streams.yaml
frankctl sources sync <source-id> --timeout 600

6. Browse datasets

frankctl datasets list --layer bronze
frankctl datasets preview bronze.tenant_00000000_local_postgres.raw_customers --limit 20

The exact dataset ID depends on tenant ID, source name, stream name, and target configuration.

7. Build a transform

Use the UI for the full guided flow:

1. Open http://localhost:5175/transforms/new.
2. Select one or more source tables.
3. Pick a target schema from FIWARE or custom schemas.
4. Accept or edit AI-suggested field mappings.
5. Preview, hydrate, and run.

Or trigger an existing transform from the CLI:

frankctl transforms list
frankctl transforms trigger <transform-id>
frankctl transforms runs <transform-id>

8. Compose a pipeline

The UI flow lives at /pipelines/new. The CLI can validate an existing pipeline:

frankctl pipelines list
frankctl pipelines validate <pipeline-id> --timeout 600

Pipeline validation starts a sandbox workflow and returns a final JSON summary.

Troubleshooting

Symptom	Check
API does not start	make logs-api, Postgres credentials, and ../common-infra status
Source discovery fails	make logs, discovery-worker logs, role-specific broker socket directory, and executor probe evidence; never add a Docker socket fallback
UI cannot reach API	VITE_API_URL, CORS origins, API port 8002
AI action returns an API error	Martha health, MARTHA_API_URL, MARTHA_CLIENT_SECRET, workflow seeding
Ontology sync reports an error	ONTOLOGY_ENABLED, ONTOLOGY_SERVICE_URL, ontology auth env vars

What is next

• Sources for EL configuration.
• Transforms for mappings, artifacts, and runs.
• AI assistance for Martha-backed help.
• Ontology integration for backing datasets.