AWS Marketplace
See our AWS Marketplace Seller Profile to purchase Kpow on AWS.
For a complete, step-by-step walkthrough, see our guide. It covers the full process, from provisioning an EKS cluster to installing Kpow with Helm using your AWS Marketplace subscription.
Read the guide: Deploy Kpow on EKS via AWS Marketplace using Helm.
Kpow integrates seamlessly with Amazon MSK and is easy to deploy on ECS, Fargate, EKS, or EKS Anywhere.
Subscribe via the AWS Marketplace to get an automatically licensed container, with billing handled directly through your AWS account.
We release every Kpow update to the AWS Marketplace — so staying up to date is as simple as pulling the latest container.
Kpow on the AWS Marketplace
When you subscribe to a Kpow product on the AWS Marketplace you gain access to a Kpow AWS Marketplace container that is functionally the same as the Kpow container available via Dockerhub.
The Kpow AWS Marketplace container integrates with AWS, allowing AWS to check your subscription, meter your usage, and checkout entitlements, depending on the version of Kpow you choose to purchase on the AWS Marketplace.
The Kpow AWS Marketplace container is automatically licensed to the AWS account that subscribes to a product and that account is billed for the subscription. You do not need to arrange a separate license with us directly if you subscribe to a Kpow product on the AWS Marketplace. It just works.
AWS Enterprise Discount Program
Kpow for Apache Kafka (Annual) can be purchased within an AWS Enterprise Discount Program (EDP).
Kpow for Apache Kafka (Hourly) does not qualify for EDP purchases.
EDP applicable purchases require a private offer, contact sales@factorhouse.io for assistance.
Kpow products on the AWS Marketplace
There are two Kpow products on the AWS Marketplace.
Kpow for Apache Kafka (Annual)
Subscribe to Kpow Annual to gain access to a Team or Enterprise Edition Kpow container that can connect to any number of Kafka clusters.
Kpow Annual is integrated with AWS License Manager. Select the number and type of Cluster credits you require when you subscribe to this product, then install as many Kpow instances as you like. Your license entitlements are checked automatically.
Kpow for Apache Kafka (Hourly)
Subscribe to Kpow Hourly to gain access to a Team Edition Kpow container that can connect to a single Kafka cluster and associated Schema Registries, Kafka Connect, and ksqlDB resources.
Start as many instances of Kpow as you need from the provided container. Usage of each instance is metered and billed by the hour at the Kpow metered rate.
Kpow Hourly allows you to pay only for the Kpow hours you use with no ongoing commitment.
The Kpow marketplace container
Getting started is easy, just choose a product from our Marketplace Seller Profile:
- Selected a product and click 'Continue to Subscribe'
- Review and accept the Terms and Conditions
- Once your subscription is activated, click 'Continue to Configuration'
- Choose the delivery method, and the latest software version available
- Click 'Continue to Launch'
- Copy the instructions to pull the Kpow AWS Marketplace container image
You can now push the Kpow container docker image into your own docker repository, or use it directly in a ECS / Fargate / EKS to launch Kpow instances with the correct IAM permissions or LM integrations.
Check your subscription regularly for new releases of Kpow and update accordingly!
General technical requirements
Target platforms
Kpow (Hourly) containers can be run in ECS (including Fargate), or EKS.
Kpow (Annual) containers can be run in ECS (including Fargate), EKS, or EKS Anywhere.
Deploy Kpow on ECS/Fargate
- Kpow Annual
- Kpow Hourly
Kpow Annual must run with the AWSLicenseManagerConsumptionPolicy IAM policy.
This policy grants the permissions required to access AWS License Manager API actions that manage and consume license entitlements.
You can attach the AWSLicenseManagerConsumptionPolicy
managed policy to either the ECS/Fargate task role used to run the Marketplace container obtained from your subscription.
Kpow Hourly must run with the AWSMarketplaceMeteringRegisterUsage IAM policy.
This policy allows the Kpow container to call the aws-marketplace:RegisterUsage
API, which is required for AWS Marketplace hourly metering.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"aws-marketplace:RegisterUsage"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
Attach this policy to the ECS/Fargate task role used to run the Marketplace container obtained from your subscription.
Deploy Kpow on EKS
Prerequisites
The minimum information Flex requires to operate is:
- License Details: No license required—billing is handled automatically through your AWS account.
- Kafka Bootstrap URL
See the Kpow Documentation for a full list of configuration options.
Create a Service Account with IAM permissions
- Kpow Annual
- Kpow Hourly
eksctl create iamserviceaccount \
--name kpow \
--namespace factorhouse \
--cluster <ENTER_YOUR_CLUSTER_NAME_HERE> \
--attach-policy-arn arn:aws:iam::aws:policy/service-role/AWSLicenseManagerConsumptionPolicy \
--approve \
--override-existing-serviceaccounts
You can now deploy Kpow to EKS using this Service Account, which includes an IAM Role with the AWSLicenseManagerConsumptionPolicy policy attached.
eksctl create iamserviceaccount \
--name kpow \
--namespace factorhouse \
--cluster <ENTER_YOUR_CLUSTER_NAME_HERE> \
--attach-policy-arn arn:aws:iam::aws:policy/AWSMarketplaceMeteringRegisterUsage \
--approve \
--override-existing-serviceaccounts
You can now deploy Kpow to EKS using this Service Account, which includes an IAM Role with the AWSMarketplaceMeteringRegisterUsage policy attached.
Configure Kubernetes/EKS
You need to connect to a Kubernetes environment before you can install Kpow.
The following examples demonstrate installing Kpow in Amazon EKS.
aws eks --region <your-aws-region> update-kubeconfig --name <your-eks-cluster-name>
Updated context arn:aws:eks:<your-aws-region>:123123123:cluster/<your-eks-cluster-name> in /your/.kube/config
You can verify that your Kubernetes cluster is available by running the following command:
kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-192-168-...-21.ec2.internal Ready <none> 2m15s v1.32.9-eks-113cf36
...
Run Kpow in Kubernetes
Setup Kpow Helm chart
- Kpow Annual
- Kpow Hourly
Download and extract the Helm chart from the Marketplace listing repository.
export HELM_EXPERIMENTAL_OCI=1
aws ecr get-login-password \
--region us-east-1 | helm registry login \
--username AWS \
--password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
mkdir awsmp-chart && cd awsmp-chart
helm pull oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/factor-house/kpow-annual-chart \
--version <VERSION_NUMBER>
tar xf $(pwd)/* && find $(pwd) -maxdepth 1 -type f -delete
Add the Factor House Helm Repository in order to use the Kpow Helm Chart.
helm repo add factorhouse https://charts.factorhouse.io
Update Helm repositories to ensure you install the latest version of Kpow.
helm repo update
Start a Kpow instance
Start Kpow with config from '--set env.XYZ'
When using helm install
, you can pass configuration with the --set env.XYZ
flag. This requires careful formatting for certain values.
Some fields, particularly integers and strings containing quotation marks, require quoting. You may also need to escape special characters (like commas or nested quotes) with a backslash (\
). For more details, see Helm's documentation on The Format and Limitations of --set
.
The following example shows how to install Kpow from the command line, highlighting how to handle escaped commas and quotes:
- Kpow Annual
- Kpow Hourly
helm install kpow ./kpow-aws-annual/ \
--set serviceAccount.create=false \
--set serviceAccount.name=kpow \
--set env.BOOTSTRAP="b-1.<cluster-name>.<cluster-identifier>.c8.kafka.us-east-1.amazonaws.com:9096" \
--set env.SECURITY_PROTOCOL="SASL_PLAINTEXT" \
--set env.SASL_MECHANISM="PLAIN" \
--set env.SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required username=\"user\" password=\"secret\";" \ # <-- note the escaped quotes
--create-namespace --namespace factorhouse
NAME: kpow
LAST DEPLOYED: Mon May 31 17:22:21 2021
NAMESPACE: factorhouse
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace factorhouse -l "app.kubernetes.io/name=kpow,app.kubernetes.io/instance=kpow" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:3000 to use your application"
kubectl --namespace factorhouse port-forward $POD_NAME 3000:3000
helm install kpow factorhouse/kpow-aws-hourly \
--set serviceAccount.create=false \
--set serviceAccount.name=kpow \
--set env.BOOTSTRAP="b-1.<cluster-name>.<cluster-identifier>.c8.kafka.us-east-1.amazonaws.com:9096" \
--set env.SECURITY_PROTOCOL="SASL_PLAINTEXT" \
--set env.SASL_MECHANISM="PLAIN" \
--set env.SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required username=\"user\" password=\"secret\";" \ # <-- note the escaped quotes
--create-namespace --namespace factorhouse
NAME: kpow
LAST DEPLOYED: Mon May 31 17:22:21 2021
NAMESPACE: factorhouse
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace factorhouse -l "app.kubernetes.io/name=kpow,app.kubernetes.io/instance=kpow" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:3000 to use your application"
kubectl --namespace factorhouse port-forward $POD_NAME 3000:3000
Start Kpow with environment variables from a ConfigMap
You can configure Kpow with a ConfigMap of environment variables as follows:
- Kpow Annual
- Kpow Hourly
helm install kpow ./kpow-aws-annual/ \
--set envFromConfigMap=kpow-config \
--create-namespace --namespace factorhouse
helm install kpow factorhouse/kpow-aws-hourly \
--set envFromConfigMap=kpow-config \
--create-namespace --namespace factorhouse
This approach requires a ConfigMap
named kpow-config
to already exist in the factorhouse
namespace. To configure Kpow with a local ConfigMap template, see Configuring with an Existing ConfigMap.
For general guidance, see the Kubernetes documentation on configuring all key-value pairs in a ConfigMap as environment variables.
Manage a Kpow instance
Set the $POD_NAME variable and test the Kpow UI
Follow the notes instructions to set the $POD_NAME variable and configure port forwarding to the Kpow UI.
export POD_NAME=$(kubectl get pods --namespace factorhouse -l "app.kubernetes.io/name=kpow,app.kubernetes.io/instance=kpow" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:3000 to use your application"
kubectl --namespace factorhouse port-forward $POD_NAME 3000:3000
Kpow is now available on http://127.0.0.1:3000.
Check the Kpow Pod
kubectl describe pods --namespace factorhouse
Name: kpow-9988df6b6-vvf8z
Namespace: factorhouse
Priority: 0
Node: ip-172-31-33-42.ap-southeast-2.compute.internal/172.31.33.42
Start Time: Mon, 31 May 2021 17:22:22 +1000
Labels: app.kubernetes.io/instance=kpow
app.kubernetes.io/name=kpow
pod-template-hash=9988df6b6
Annotations: kubernetes.io/psp: eks.privileged
Status: Running
View the Kpow Pod logs
kubectl logs $POD_NAME --namespace factorhouse
11:36:49.111 INFO [main] kpow.system ? start Kpow
...
Remove Kpow
helm delete kpow --namespace factorhouse
Start Kpow with local changes
- Kpow Annual
- Kpow Hourly
You can run Kpow with local edits to chart files to provide custom configuration.
Make local edits
Make any edits required to kpow-aws-annual/Chart.yaml
or kpow-aws-annual/values.yaml
(adding volume mounts, etc).
Configuring with an existing ConfigMap
This is the recommended method for managing configuration separately from the Helm chart.
1. Prepare Your ConfigMap Manifest
Copy the example file (kpow-config.yaml.example), then edit it to set your desired metadata.name
(e.g., kpow-config
) and fill in your configuration under the data
section.
cp ./kpow-aws-annual/kpow-config.yaml.example kpow-config.yaml
# now edit kpow-config.yaml
2. Create the ConfigMap in Kubernetes
Before installing, use kubectl
to create the ConfigMap
object in your cluster from the file you just prepared.
kubectl apply -f kpow-config.yaml --namespace factorhouse
3. Install the Chart
Install the Helm chart, using --set
to reference the name of the ConfigMap
you just created. The --create-namespace
flag will ensure the target namespace exists.
helm install kpow ./kpow-aws-annual \
--set envFromConfigMap=kpow-config \
--create-namespace --namespace factorhouse
The Kpow pod will now start using the environment variables from your externally managed ConfigMap
.
See kpow-config.yaml.example for an example ConfigMap file.
See the Kubernetes documentation on configuring all key-value pairs in a config map as container environment variables for more information.
You can run Kpow with local edits to chart files to provide custom configuration.
Pull and Untar the Kpow Charts
helm pull factorhouse/kpow-aws-hourly --untar --untardir .
Make Local Edits
Make any edits required to kpow-aws-hourly/Chart.yaml
or kpow-aws-hourly/values.yaml
(adding volume mounts, etc).
Run Local Charts
The command to run local charts is slightly different, see ./kpow-aws-hourly
rather than factorhouse/kpow-aws-hourly
.
helm install kpow ./kpow-aws-hourly \
<.. --set configuration, etc ..> \
--create-namespace -namespace factorhouse
Configuring with an Existing ConfigMap
This is the recommended method for managing configuration separately from the Helm chart.
1. Prepare Your ConfigMap Manifest
Copy the example file (kpow-config.yaml.example), then edit it to set your desired metadata.name
(e.g., kpow-config
) and fill in your configuration under the data
section.
cp ./kpow-aws-hourly/kpow-config.yaml.example kpow-config.yaml
# now edit kpow-config.yaml
2. Create the ConfigMap in Kubernetes
Before installing, use kubectl
to create the ConfigMap
object in your cluster from the file you just prepared.
kubectl apply -f kpow-config.yaml --namespace factorhouse
3. Install the Chart
Install the Helm chart, using --set
to reference the name of the ConfigMap
you just created. The --create-namespace
flag will ensure the target namespace exists.
helm install kpow ./kpow-aws-hourly \
--set envFromConfigMap=kpow-config \
--create-namespace --namespace factorhouse
The Kpow pod will now start using the environment variables from your externally managed ConfigMap
.
See kpow-config.yaml.example for an example ConfigMap file.
See the Kubernetes documentation on configuring all key-value pairs in a config map as container environment variables for more information.
Manage sensitive environment variables
This helm chart accepts the name of a secret containing sensitive parameters, e.g.
apiVersion: v1
kind: Secret
metadata:
name: kpow-secrets
data:
SASL_JAAS_CONFIG: a3JnLmFwYWNoXS5rYWZrYS5jb21tb24uc2VjdXJpdHkucGxhaW4uUGxhaW5Mb2dpbk2vZHVsZSByZXF1aXJiZCB1c2VybmFtZT0iTFQ1V0ZaV1BRWUpHNzRJQyIgcGFzc3dvcmQ9IjlYUFVYS3BLYUQxYzVJdXVNRjRPKzZ2NxJ0a1E4aS9yWUp6YlppdlgvZnNiTG51eGY4SnlFT1dUeXMvTnJ1bTAiBwo=
CONFLUENT_API_SECRET: NFJSejlReFNTTXlTcGhXdjNLMHNYY1F6UGNURmdadlNYT0ZXSXViWFJySmx2N3A2WStSenROQnVpYThvNG1NSRo=
kubectl apply -f ./kpow-secrets.yaml --namespace factorhouse
Then run the helm chart (this can be used in conjunction with envFromConfigMap
)
See the Kubernetes documentation on configuring all key value pairs in a secret as environment variables for more information.
- Kpow Annual
- Kpow Hourly
helm install kpow ./kpow-aws-annual/ \
--set envFromSecret=kpow-secrets \
--set envFromConfigMap=kpow-config \
--create-namespace --namespace factorhouse
helm install kpow ./kpow-aws-hourly/ \
--set envFromSecret=kpow-secrets \
--set envFromConfigMap=kpow-config \
--create-namespace --namespace factorhouse
Provide files to the Kpow Pod
There are occasions where you must provide files to the Kpow Pod in order for Kpow to run correctly, such files include:
- RBAC configuration
- SSL Keystores
- SSL Truststores
How you provide these files is down to user preference, we are not able to provide any support or instruction in this regard.
You may find the Kubernetes documentation on injecting data into applications useful.
Kpow memory and CPU requirements
The chart runs Kpow with Guaranteed QoS, having resource request and limit set to these values by default:
resources:
limits:
cpu: 2
memory: 8Gi
requests:
cpu: 2
memory: 8Gi
These default resource settings are conservative, suited to a deployment of Kpow that manages multiple Kafka clusters and associated resources.
When running Kpow with a single Kafka cluster you can experiment with reducing those resources as far as our suggested minimum:
Minimum resource requirements
resources:
limits:
cpu: 1
memory: 2Gi
requests:
cpu: 1
memory: 2Gi
Adjust these values from the command line like so:
- Kpow Annual
- Kpow Hourly
helm install kpow ./kpow-aws-annual/ \
--set resources.limits.cpu=1 \
--set resources.limits.memory=2Gi \
--set resources.requests.cpu=1 \
--set resources.requests.memory=2Gi \
--create-namespace --namespace factorhouse
helm install kpow ./kpow-aws-hourly/ \
--set resources.limits.cpu=1 \
--set resources.limits.memory=2Gi \
--set resources.requests.cpu=1 \
--set resources.requests.memory=2Gi \
--create-namespace --namespace factorhouse
We recommend always having limits and requests set to the same value, as this set Kpow in Guaranteed QoS and provides a much more reliable operation.
Snappy compression in read-only filesystem
We preset an attribute for Snappy compression in read-only filesystems. It is disabled by default and can be enabled - modify the volume configuration if necessary.
ephemeralTmp:
enabled: true
volume:
emptyDir:
medium: Memory # Optional: for better performance
sizeLimit: "100Mi" # Configurable size
Deploy Kpow Annual on EKS Anywhere
The Kpow Annual Helm chart includes extra resources required for the token-based IAM authentication used by EKS Anywhere. It can be configured as follows.
Create token & IAM role
- In the AWS Marketplace console, create a license token and an associated IAM role for the Kpow subscription.
- This token is used to access AWS License Manager APIs for license validation.
- A button to generate these is available after you subscribe to the product.
Configure Kubernetes Secrets and Service Account
1. Create the namespace and a dedicated service account
kubectl create namespace factorhouse
kubectl create serviceaccount kpow --namespace factorhouse
2. Create the license secret with the values from Step 1
# IMPORTANT: Replace the placeholder values below with your actual token and role ARN.
AWSMP_TOKEN="<YOUR_LICENSE_TOKEN_HERE>"
AWSMP_ROLE_ARN="<YOUR_IAM_ROLE_ARN_HERE>"
kubectl create secret generic awsmp-license-token-secret \
--from-literal=license_token=$AWSMP_TOKEN \
--from-literal=iam_role=$AWSMP_ROLE_ARN \
--namespace factorhouse
3. Create an ECR image pull secret using the license token
AWSMP_ACCESS_TOKEN=$(aws license-manager get-access-token \
--output text --query '*' --token $AWSMP_TOKEN --region us-east-1)
AWSMP_ROLE_CREDENTIALS=$(aws sts assume-role-with-web-identity \
--region 'us-east-1' \
--role-arn $AWSMP_ROLE_ARN \
--role-session-name 'AWSMP-guided-deployment-session' \
--web-identity-token $AWSMP_ACCESS_TOKEN \
--query 'Credentials' \
--output text)
export AWS_ACCESS_KEY_ID=$(echo $AWSMP_ROLE_CREDENTIALS | awk '{print $1}' | xargs)
export AWS_SECRET_ACCESS_KEY=$(echo $AWSMP_ROLE_CREDENTIALS | awk '{print $3}' | xargs)
export AWS_SESSION_TOKEN=$(echo $AWSMP_ROLE_CREDENTIALS | awk '{print $4}' | xargs)
kubectl create secret docker-registry awsmp-image-pull-secret \
--docker-server=709825985650.dkr.ecr.us-east-1.amazonaws.com \
--docker-username=AWS \
--docker-password=$(aws ecr get-login-password --region us-east-1) \
--namespace factorhouse
4. Link the image pull secret to the service account
kubectl patch serviceaccount kpow \
--namespace factorhouse \
-p '{"imagePullSecrets": [{"name": "awsmp-image-pull-secret"}]}'
Launch Kpow Annual chart
Download and extract the Helm chart from the Marketplace listing repository.
export HELM_EXPERIMENTAL_OCI=1
aws ecr get-login-password \
--region us-east-1 | helm registry login \
--username AWS \
--password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com
mkdir awsmp-chart && cd awsmp-chart
helm pull oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/factor-house/kpow-aws-annual \
--version <VERSION_NUMBER>
tar xf $(pwd)/* && find $(pwd) -maxdepth 1 -type f -delete
Install Kpow, referencing the Kubernetes resources you created above.
helm install kpow ./kpow-aws-annual/ \
--set serviceAccount.create=false \
--set serviceAccount.name=kpow \
--set aws.licenseConfigSecretName=awsmp-license-token-secret \
--create-namespace --namespace factorhouse
Get help!
For assistance installing and configuring Kpow on the AWS Marketplace contact support@factorhouse.io