Skip to main content
CommunityTeamEnterprise

MSK Connect

Kpow provides comprehensive support for Amazon MSK Connect, allowing you to seamlessly monitor and manage your Kafka connectors within the AWS ecosystem. Kpow integrates directly with the MSK Connect API to provide a real-time view of your connector and task statuses, configurations, and performance metrics.

IAM access control​

Kpow authenticates with Amazon MSK Connect by using the AWS SDK. You can configure authentication using one of the following methods, which determine the IAM principal Kpow will use to make API calls.

Default credentials provider chain​

By default, Kpow will use the DefaultAWSCredentialsProviderChain to authenticate with Amazon MSK connect. This is the recommended approach for resources running within AWS (e.g., on EC2, ECS, or EKS), as Kpow can automatically and securely inherit permissions from an attached IAM Role.

Static credentials​

You can authenticate using static credentials for a specific IAM user by specifying its access keys. This method is often used for local development or for environments outside of AWS.

CONNECT_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
CONNECT_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Cross-account AWS access (STS AssumeRole)​

If you wish to configure cross-account Amazon MSK Connect access (where Kpow is running in AWS Account A and Amazon MSK Connect is in Account B), you can configure an IAM role and a corresponding trust policy.

You can then configure Kpow to assume that role with the following environment variables:

CONNECT_STS_ROLE_ARN="arn:aws:iam::ACCOUNT_B_ID:role/KpowMskConnectAccess"
CONNECT_STS_SESSION_NAME="kpow-session"
CONNECT_STS_REGION="us-east-2" # (default to us-east-1)
tip

To manage multiple cross-account environments, you must provide a complete set of STS variables for each additional connection, using a numeric suffix (e.g., CONNECT_STS_ROLE_ARN_2, CONNECT_STS_ROLE_ARN_3). Ensure the suffix _N matches the corresponding Kafka cluster (BOOTSTRAP_N).

MSK Connect AWS region​

It is required to specify the AWS region where your MSK Connect cluster is deployed.

CONNECT_AWS_REGION=us-east-1
tip

When configuring multiple MSK Connect environments, each one requires its own region definition using a numeric suffix (e.g., CONNECT_AWS_REGION_2, CONNECT_AWS_REGION_3). Kpow uses this suffix to associate the Connect instance with the correctly numbered Kafka cluster.

Example IAM policies for Kpow​

Once authenticated, the IAM principal used by Kpow requires permissions to interact with the MSK Connect service. Below are policy examples that parallel the access patterns for MSK clusters.

Option 1: Admin access to all MSK Connect resources in the region/account

This policy allows listing all resources and performing any administrative action (e.g., create, delete, update) on any connector, custom plugin, or worker configuration within the account.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kafkaconnect:CreateConnector",
        "kafkaconnect:DeleteConnector",
        "kafkaconnect:DescribeConnector",
        "kafkaconnect:UpdateConnector",
        "kafkaconnect:StopConnector",
        "kafkaconnect:StartConnector",
        "kafkaconnect:DescribeCustomPlugin",
        "kafkaconnect:DescribeWorkerConfiguration"
      ],
      "Resource": "arn:aws:kafkaconnect:<REGION>:<ACCOUNT-ID>:*/*/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kafkaconnect:ListConnectors",
        "kafkaconnect:ListCustomPlugins",
        "kafkaconnect:ListWorkerConfigurations"
      ],
      "Resource": "*"
    }
  ]
}

Option 2: Admin access to a specific list of connectors

This policy allows Kpow to discover all MSK Connect resources but restricts administrative actions to a specific list of connectors. This is the recommended approach for production environments where Kpow should only manage a subset of connectors.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kafkaconnect:DeleteConnector",
                "kafkaconnect:UpdateConnector",
                "kafkaconnect:StopConnector",
                "kafkaconnect:StartConnector"
            ],
            "Resource": [
                "arn:aws:kafkaconnect:<REGION>:<ACCOUNT-ID>:connector/<CONNECTOR-NAME-1>/<GUID-1>",
                "arn:aws:kafkaconnect:<REGION>:<ACCOUNT-ID>:connector/<CONNECTOR-NAME-2>/<GUID-2>"
                // Add more connector ARNs here as needed following the same pattern
                // "arn:aws:kafkaconnect:<REGION>:<ACCOUNT-ID>:connector/<CONNECTOR-NAME-3>/<GUID-3>"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafkaconnect:ListConnectors",
                "kafkaconnect:DescribeConnector",
                "kafkaconnect:ListCustomPlugins",
                "kafkaconnect:DescribeCustomPlugin",
                "kafkaconnect:ListWorkerConfigurations",
                "kafkaconnect:DescribeWorkerConfiguration"
            ],
            "Resource": "*"
        }
    ]
}