MSK Connect

Example IAM policy

Configure Kpow with an IAM policy similar to the one below:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "KafkaConnect",
      "Effect": "Allow",
      "Action": [
        "kafkaconnect:DeleteConnector",
        "kafkaconnect:ListConnectors",
        "kafkaconnect:ListCustomPlugins",
        "kafkaconnect:ListWorkerConfigurations"
      ],
      "Resource": "arn:${Partition}:kafkaconnect:${Region}:${Account}:*"
    },
    {
      "Sid": "Connector",
      "Effect": "Allow",
      "Action": [
        "kafkaconnect:DescribeConnector"
      ],
      "Resource": "arn:${Partition}:kafkaconnect:${Region}:${Account}:connector/*/*"
    },
    {
      "Sid": "CustomPlugin",
      "Effect": "Allow",
      "Action": [
        "kafkaconnect:DescribeCustomPlugin"
      ],
      "Resource": "arn:${Partition}:kafkaconnect:${Region}:${Account}:custom-plugin/*/*"
    },
    {
      "Sid": "WorkerConfiguration",
      "Effect": "Allow",
      "Action": [
        "kafkaconnect:DescribeWorkerConfiguration"
      ],
      "Resource": "arn:${Partition}:kafkaconnect:${Region}:${Account}:worker-configuration/*/*"
    }
  ]
}

You can learn more about Kafka Connect IAM actions and resources at the official Amazon documentation.

Kpow configuration

Specify the AWS region your MSK connectors/cluster belong to:

CONNECT_AWS_REGION=us-east-1

Note: By default kpow assumes the MSK connect cluster lives in the first defined kafka cluster. Make sure the kafka connect suffix matches CONNECT_AWS_REGION_N matches the kafka cluster suffix, ie BOOTSTRAP_N.

MSK Connect authentication

By default, Kpow will use the DefaultAWSCredentialsProviderChain to authenticate with AWS MSK connect.

Static credentials

You can authenticate using static credentials by specifying:

CONNECT_ACCESS_KEY_ID=xxx
CONNECT_SECRET_ACCESS_KEY=xxx

Cross-account AWS Access (STS Assume Role)

If you wish to configure cross-account MSK Connect access (where for example Kpow is living in AWS Account A and MSK Connect is living in Account B) you can configure an IAM role+trust policy.

You can configure Kpow to assume a role with the following environment variables:

CONNECT_STS_ROLE_ARN="arn:aws:iam::ACCOUNT_B_ID:root"
CONNECT_STS_SESSION_NAME="session_name"
CONNECT_STS_REGION="us-east-2" # (default us-east-1)