Frank

Appearance

API Reference ​

Frank exposes a FastAPI API under /api/v1. The interactive OpenAPI UI is available at /docs when the API is running.

Local default:

text
http://localhost:8002/docs

Health and platform status ​

http
GET /health
GET /api/v1/status
GET /api/v1/services/health
GET /api/v1/metrics

Use these for readiness checks, service health, and metrics.

Sources ​

http
POST   /api/v1/sources/discover
GET    /api/v1/sources/discover/status/{workflow_id}
GET    /api/v1/sources
POST   /api/v1/sources
GET    /api/v1/sources/{source_id}
PATCH  /api/v1/sources/{source_id}
DELETE /api/v1/sources/{source_id}
POST   /api/v1/sources/{source_id}/sync
GET    /api/v1/sources/{source_id}/sync/status
GET    /api/v1/sources/{source_id}/streams
POST   /api/v1/sources/{source_id}/streams
PATCH  /api/v1/sources/{source_id}/streams/{stream_id}
DELETE /api/v1/sources/{source_id}/streams/{stream_id}
POST   /api/v1/sources/{source_id}/streams/bulk
POST   /api/v1/sources/{source_id}/streams/refresh-schema
GET    /api/v1/sources/{source_id}/sync-history
GET    /api/v1/sources/{source_id}/sync-history/stats
GET    /api/v1/sources/{source_id}/iceberg-refs
GET    /api/v1/sources/{source_id}/runs/{run_id}/logs
GET    /api/v1/sources/{source_id}/runs/{run_id}/logs/download
GET    /api/v1/sources/sync-runs/{sync_run_id}/streams
GET    /api/v1/sources/sync-runs/{workflow_id}/details
GET    /api/v1/sources/{source_id}/credentials
POST   /api/v1/sources/{source_id}/credentials
PUT    /api/v1/sources/{source_id}/credentials
POST   /api/v1/sources/{source_id}/credentials/migrate
DELETE /api/v1/sources/{source_id}/credentials
POST   /api/v1/sources/credentials/{credential_ref}/revoke

Declared credential fields are write-only in both server modes. Source readback, dry-run, export, logs, and errors never return their values.

Current mutable/base Compose uses SOURCE_CREDENTIAL_MODE=legacy_inline. Source create/update accepts credential_values, merges them into Source.source_config, and preserves stored credential fields on ordinary updates even though readback is redacted. credential_ref is unavailable. Every /credentials lifecycle endpoint listed above fails clearly in this mode; discovery and extraction continue to use the inline Source configuration.

The immutable release uses SOURCE_CREDENTIAL_MODE=vault. Source create/update accepts either credential_values plus credential_name, or an existing tenant-owned credential_ref, never both. Credential bootstrap and rotation responses contain only safe metadata: opaque reference, name, kind, declared field names, presence, exact version, lifecycle status, rotation time, and reference count. Values and Vault paths are never returned.

The migrate route writes and verifies Vault before removing declared fields from PostgreSQL. Detach never deletes or revokes Vault data. Revoke is allowed only after every Source has detached and retains Vault history for operator-managed recovery.

POST .../streams/refresh-schema accepts { "streams": [...] } as a complete discovery snapshot. Existing streams are matched by name; only schema and supported_sync_modes are refreshed. Namespace/destination routing, sync/write/cursor/enabled configuration, and runtime state are preserved. New names are created with defaults and names absent from a non-empty snapshot are removed as stale. A discovery namespace must identify a real upstream container such as a database schema or GeoServer workspace; namespace-less API streams use null, never their own stream name.

Source schedules ​

http
GET    /api/v1/sources/{source_id}/schedule
PUT    /api/v1/sources/{source_id}/schedule
POST   /api/v1/sources/{source_id}/schedule/pause
POST   /api/v1/sources/{source_id}/schedule/resume
POST   /api/v1/sources/{source_id}/schedule/trigger
DELETE /api/v1/sources/{source_id}/schedule
GET    /api/v1/schedules

Source patterns ​

http
GET /api/v1/patterns/discover
GET /api/v1/patterns/{pattern_id}

External pattern lifecycle:

http
POST /api/v1/patterns/register
POST /api/v1/patterns/reconcile

Transforms ​

http
GET    /api/v1/transforms
GET    /api/v1/transforms/{transform_id}
GET    /api/v1/transforms/{transform_id}/detail
PATCH  /api/v1/transforms/{transform_id}
DELETE /api/v1/transforms/{transform_id}
POST   /api/v1/transforms/{transform_id}/hydrate
POST   /api/v1/transforms/{transform_id}/execute
POST   /api/v1/transforms/{transform_id}/materialize
GET    /api/v1/transforms/{transform_id}/runs
GET    /api/v1/transforms/{transform_id}/runs/{run_id}

/execute is the low-level execution callback path. Product and CLI execution use the Dagster-backed materialization or schedule trigger paths.

Transform schedules ​

http
GET    /api/v1/transforms/{transform_id}/schedule
PUT    /api/v1/transforms/{transform_id}/schedule
POST   /api/v1/transforms/{transform_id}/schedule/pause
POST   /api/v1/transforms/{transform_id}/schedule/resume
POST   /api/v1/transforms/{transform_id}/schedule/trigger
DELETE /api/v1/transforms/{transform_id}/schedule
GET    /api/v1/transform-schedules

Transform patterns and SQL transforms ​

http
GET  /api/v1/transform-patterns
GET  /api/v1/transform-patterns/categories
GET  /api/v1/transform-patterns/{pattern_id}
POST /api/v1/transform-patterns/{pattern_id}/validate
POST /api/v1/transform-patterns/{pattern_id}/preview
POST /api/v1/transform-patterns/sync
http
GET  /api/v1/sql-transforms
GET  /api/v1/sql-transforms/categories
GET  /api/v1/sql-transforms/for-type/{sql_type}
GET  /api/v1/sql-transforms/{transform_id}
POST /api/v1/sql-transforms/{transform_id}/apply
POST /api/v1/sql-transforms/sync

Artifacts, runs, and logs ​

http
GET /api/v1/artifacts
GET /api/v1/artifacts/{artifact_id}
GET /api/v1/artifacts/{artifact_id}/content
GET /api/v1/artifacts/{artifact_id}/download
http
POST /api/v1/runs
POST /api/v1/runs/{id}/start
POST /api/v1/runs/{id}/complete
POST /api/v1/runs/{id}/fail
POST /api/v1/runs/{id}/cancel
GET  /api/v1/runs/{id}
GET  /api/v1/runs
http
GET  /api/v1/transform-runs
GET  /api/v1/transform-runs/{run_id}
GET  /api/v1/transform-runs/{run_id}/logs
POST /api/v1/transform-runs/{run_id}/cancel

Pipelines ​

http
POST   /api/v1/pipelines
GET    /api/v1/pipelines
GET    /api/v1/pipelines/{pipeline_id}
PUT    /api/v1/pipelines/{pipeline_id}
PATCH  /api/v1/pipelines/{pipeline_id}/steps/{step_id}
DELETE /api/v1/pipelines/{pipeline_id}
POST   /api/v1/pipelines/validate-dag
POST   /api/v1/pipelines/{pipeline_id}/versions
POST   /api/v1/pipelines/{pipeline_id}/activate
POST   /api/v1/pipelines/{pipeline_id}/pause
POST   /api/v1/pipelines/{pipeline_id}/sandbox
GET    /api/v1/pipelines/{pipeline_id}/sandbox/{workflow_id}/status
GET    /api/v1/pipelines/{pipeline_id}/runs
GET    /api/v1/pipelines/available-transforms
POST   /api/v1/apply/dry-run

POST /api/v1/apply/dry-run accepts the same ordered envelope documents used by declarative apply:

json
{
  "docs": [
    {
      "apiVersion": "frank.platform/v1",
      "kind": "Source",
      "metadata": {"name": "example"},
      "spec": {"pattern_id": "rest_api", "source_config": {}}
    }
  ],
  "allow_deprecate": false,
  "live_ontology": false
}

The route never persists. Each result reports create, patch, noop, would-409, or error; Source results also include stream-level create/patch/noop/unmanaged actions. Set live_ontology: true to add GET-only validation of each locally valid BackingDataset against its declared ontology tenant, type, properties, relationships, keys, and carrier types. Network/authentication failures make that preflight invalid. The live check does not create or modify schema or entities.

Registered declarative routes also accept Content-Type: application/yaml. Unquoted YAML dates and datetimes are recursively converted to ISO-8601 strings before normal JSON validation. JSON-compatible scalar values retain their types. YAML-only values that cannot be represented as JSON return a sanitized HTTP 400 without echoing the submitted spec or credential-shaped values.

Datasets ​

http
GET /api/v1/datasets
GET /api/v1/datasets/{dataset_id}/preview
GET /api/v1/datasets/{dataset_id}/snapshots
GET /api/v1/datasets/{dataset_id}

dataset_id is the dotted table identifier accepted by the route, for example silver.tenant_x.table_name.

AI ​

http
POST /api/v1/ai/suggest-target-schema
POST /api/v1/ai/suggest-field-mappings
POST /api/v1/ai/suggest-pattern-params
POST /api/v1/ai/review-sql
POST /api/v1/ai/generate-transform
POST /api/v1/ai/fix-ci-failure
POST /api/v1/ai/publish-transform
POST /api/v1/ai/compose-pipeline
GET  /api/v1/ai/compose-pipeline/{execution_id}/status

Schema libraries and SDM registry ​

http
GET  /api/v1/schema-libraries
GET  /api/v1/schema-libraries/{library_id}/domains
GET  /api/v1/schema-libraries/{library_id}/domains/{domain}/schemas
GET  /api/v1/schema-libraries/{library_id}/schemas/{schema_id}
GET  /api/v1/schema-libraries/schema/{full_id}
GET  /api/v1/schema-libraries/search
POST /api/v1/schema-libraries/validate/{full_id}
http
GET  /api/v1/sdm
GET  /api/v1/schemas/registry
POST /api/v1/schemas/registry/register
POST /api/v1/schemas/registry/bulk-register
POST /api/v1/schemas/registry/discover-and-register
POST /api/v1/schemas/registry/sync-catalog
POST /api/v1/schemas/registry/search
POST /api/v1/schemas/registry/validate

Ontology and backing datasets ​

http
GET    /api/v1/ontology/status
GET    /api/v1/ontology/entity-types
GET    /api/v1/ontology/entity-types/domains
GET    /api/v1/ontology/entity-types/{code}
POST   /api/v1/ontology/entity-types
POST   /api/v1/ontology/entity-types/{code}/versions
PATCH  /api/v1/ontology/entity-types/{code}
DELETE /api/v1/ontology/entity-types/{code}
GET    /api/v1/ontology/entity-types/{code}/versions
http
POST   /api/v1/backing-datasets/suggest-mappings
GET    /api/v1/backing-datasets/pending-syncs
GET    /api/v1/backing-datasets/capabilities
POST   /api/v1/backing-datasets
GET    /api/v1/backing-datasets
GET    /api/v1/backing-datasets/{backing_dataset_id}
PATCH  /api/v1/backing-datasets/{backing_dataset_id}
DELETE /api/v1/backing-datasets/{backing_dataset_id}
POST   /api/v1/backing-datasets/{backing_dataset_id}/sync?force={boolean}
GET    /api/v1/backing-datasets/{backing_dataset_id}/sync-history
GET    /api/v1/backing-datasets/{backing_dataset_id}/sync-history/{run_id}
GET    /api/v1/backing-datasets/{backing_dataset_id}/sync-history/{run_id}/logs
GET    /api/v1/backing-datasets/{backing_dataset_id}/health

The sync trigger returns the persisted run ID, attempted effective-SyncSpec fingerprint, and force flag. Run responses expose the replay reason and both attempted/applied fingerprints without credentials or volatile workflow data in the fingerprint itself. They also expose both the legacy numeric snapshot_id and additive decimal-string snapshot_id_exact; use the exact field whenever 64-bit Iceberg snapshot precision matters, especially in JavaScript clients. The exact-run endpoint additionally returns is_terminal, rows_synced, and bulk_jobs. Each bulk evidence item contains only job_id, operation, submitted_count, and completed_count; request rows and credentials are not persisted or returned.

GET /api/v1/backing-datasets/capabilities returns:

json
{
  "sync_contract_version": 2,
  "supports_exact_run_polling": true,
  "supports_force_replay": true
}

The V2 trigger fields are optional in the response schema so a newer CLI can recognize an N-1 response shape. When present together, they are the contract for polling the exact run. Clients must require the capabilities response and force_replay: true confirmation before treating a force request as accepted.

Identity policies and users ​

http
GET    /api/v1/identity-policies
GET    /api/v1/identity-policies/{policy_id}
GET    /api/v1/identity-policies/{policy_id}/versions
GET    /api/v1/identity-policies/{policy_id}/versions/{version}
POST   /api/v1/identity-policies
POST   /api/v1/identity-policies/{policy_id}/clone
PUT    /api/v1/identity-policies/{policy_id}
DELETE /api/v1/identity-policies/{policy_id}
POST   /api/v1/identity-policies/{policy_id}/dry-run
http
GET /api/v1/users
GET /api/v1/users/{sub}

Temporal ​

http
GET  /api/v1/temporal/health
GET  /api/v1/temporal/health/detailed
GET  /api/v1/temporal/config
GET  /api/v1/temporal/metrics
GET  /api/v1/temporal/workers
GET  /api/v1/temporal/task-queues
POST /api/v1/temporal/workflows/schema-match
POST /api/v1/temporal/workflows/transform
POST /api/v1/temporal/workflows/pipeline
GET  /api/v1/temporal/workflows/{workflow_id}
POST /api/v1/temporal/workflows/{workflow_id}/cancel
POST /api/v1/temporal/workflows/{workflow_id}/wait

Frank — low-code EL/T for the lakehouse.