Data Platform

Step 01 · Ingest

Load your data

Drop files into the lake, wire up any external credentials, and confirm what landed in the warehouse.

Name	Size	Modified	Actions

Files land in the object store and are readable from KNIME via the Generic S3 Connector (endpoint + credentials: see Connection info; path-style access on). Tip: partition by date, e.g. sales/year=2026/month=07/orders.parquet.

Connection credentials

include secrets

Name	Login	Secret	Description	Updated	Actions

For external systems this data product's workflows connect to (other databases, SharePoint, APIs …). Stored encrypted; injected into runs as KNIME Workflow Credential <name> and flow variables <name>_login / <name>_password. Saving an existing name updates it (leave the secret blank to keep the stored one).

Warehouse tables

Table	Rows	Columns	Actions

Quick peek only — for charts and SQL head to the Visualize step. In KNIME, read these via PostgreSQL Connector (host postgres, database warehouse).

Step 02 · Build

Build a workflow

Author the transformation in KNIME, export a .knwf, and register it under this data product.

Upload workflow

Workflows

Workflow	Manage

Keep versions and roll back from here. Cadence, on/off and run history live in the Schedule & run step.

Step 03 · Orchestrate

Schedule & run

Set cadences, switch workflows on, trigger runs, and chain them into pipelines.

Workflow schedule

Workflow	Schedule	Enabled	Last run	Actions

Pipelines

Pipeline	Steps (stages run in order, same stage = parallel)	Schedule	Enabled	Last run	Actions

Superset — sign in with your platform account (Keycloak SSO). If the frame stays on a login screen, use “Open in new tab” once, then return. The Warehouse connection is pre-configured; tables live under Datasets → + Dataset. Open in new tab ↗

Admin · Configuration

Data products

Create a data product to provision its Keycloak group, S3 prefixes, warehouse schema and service account in one step.

Name	Warehouse schema	S3 paths	Service user	Actions

Creating a data product sets up everything at once: Keycloak group (assign users there), S3 prefixes in raw/curated, warehouse schema, and a service account the executor uses for this product's workflows. See docs/access-management.md for the full guide.