Main services

The enterprise editions of Kpow and Flex support multiple authentication providers, as outlined in the documentation below:

• Kpow Authentication Overview
• Flex Authentication Overview

Among those, this project includes file-based authentication, and user credentials can be found in the following locations:

• Kpow: resources/kpow/config/trial.env
• Flex: resources/flex/config/trial.env

For demonstration purposes, Kpow and Flex can be accessed using admin as both the username and password.

After successful authentication, users are redirected to the Overview page.

The following sections show key services and their associated port mappings.

Kafka with Kpow (KRaft Mode)

Service Name	Port(s) (Host:Container)	Description
kpow	3000:3000 4000:4000	Kpow Web UI (3000) Kpow API (4000).
schema	8081:8081	Confluent Schema Registry (manages Kafka message schemas).
connect	8083:8083	Kafka Connect (framework for Kafka connectors).
kafka-1	9092:9092	Kafka Broker 1 (primary listener).
kafka-2	9093:9093	Kafka Broker 2 (primary listener).
kafka-3	9094:9094	Kafka Broker 3 (primary listener).

Unified Analytics with Flex

Service Name	Port(s) (Host:Container)	Description
flex	3001:3000	Flex UI for enterprise Flink management and monitoring.
jobmanager	8082:8081	Apache Flink JobManager UI & REST API.
sql-gateway	9090:9090	Apache Flink SQL Gateway for submitting interactive SQL queries.
spark	4040:4040 18080:18080	Spark Web UI (4040). Spark History Server (18080).
minio	9001:9001 9000:9000	MinIO Console UI (9001). MinIO S3-compatible API (9000).
postgres	5432:5432	PostgreSQL database, used as the Hive Metastore backend.
hive-metastore	9083:9083	Apache Hive Metastore service.
redis	6379:6379	Redis in-memory data store.

__(Note: Services like taskmanager-_andmc run in the background and do not expose ports to the host.)_*

Real-time Data Stores

These services are launched from compose-store.yml using profiles.

Pinot (--profile pinot)

Service Name	Port(s) (Host:Container)	Description
zookeeper	2181:2181	ZooKeeper (coordination service, required for Pinot).
pinot-controller	19000:9000	Apache Pinot Controller (manages cluster state, UI/API).
pinot-broker	18099:8099	Apache Pinot Broker (handles query routing).
pinot-server	18098:8098	Apache Pinot Server (hosts data and executes queries).

ClickHouse (--profile clickhouse)

Service Name	Port(s) (Host:Container)	Description
ch-app	8000:8000 8084:8080	HyperDX API (8000). HyperDX Web UI (8084).
ch-server	8123:8123 29000:9000	ClickHouse Server HTTP API (8123). Native TCP client (29000).

StarRocks (--profile starrocks)

Service Name	Port(s) (Host:Container)	Description
starrocks-fe	8030:8030 9030:9030	StarRocks Frontend UI (8030) MySQL SQL client (9030).

Observability Stacks

These services are launched from compose-obsv.yml using profiles.

Lineage (--profile lineage)

Service Name	Port(s) (Host:Container)	Description
marquez-web	3003:3000	Marquez Web UI (visualizes OpenLineage data).
marquez-api	5002:5000	Marquez backend and OpenLineage API endpoint.
marquez-db	5433:5432	PostgreSQL database for Marquez metadata.

Telemetry (--profile telemetry)

Service Name	Port(s) (Host:Container)	Description
prometheus	19090:9090	Prometheus (metrics collection and database).
alertmanager	19093:9093	Alertmanager (handles alerts from Prometheus).
grafana	3004:3000	Grafana (platform for metrics visualization & dashboards).

Data Discovery & Governance (OpenMetadata)

These services are launched from compose-metadata.yml using the --profile omt flag.

Service Name	Port(s) (Host:Container)	Description
omt-server	8585:8585	OpenMetadata Web UI and central API.
omt-ingestion	8080:8080	Airflow UI for managing metadata ingestion workflows.
omt-es	9200:9200	Elasticsearch (powers the search and discovery features).
omt-db	5434:5432	PostgreSQL database for the OpenMetadata backend.