Skip to main content
CommunityTeamEnterprise

Instaclustr Platform

What is Instaclustr Platform?​

The Instaclustr platform is a unified, managed service for deploying, operating, and monitoring open-source data infrastructure technologies like Apache Kafka, Apache Cassandra, and PostgreSQL in both cloud and on-premises environments. It simplifies the complexities of these powerful open-source tools by providing a single interface for management, with 24/7 expert support, allowing companies to focus on application development rather than infrastructure maintenance.

Quickstart​

tip

This document is adapted from Set Up Kpow with Instaclustr Platform. Check out the post for more details and step-by-step instructions.

Create Kafka and Kafka Connect clusters​

Within the Instaclustr console, create a Kafka cluster. For this demonstration, include the Karapace schema registry as an enterprise add-on. When creating the Kafka Connect cluster, ensure that the previously created Kafka cluster is selected as the target. It is also crucial to enable the Use custom connectors option. This will present a configuration form to provide the name of the S3 bucket containing the connector artifacts and the necessary AWS credentials. For more detailed instructions, refer to the official Instaclustr documentation.

Update firewall rules​

When the Kafka Connect cluster is linked to the target Kafka cluster, the firewall rules for the Kafka cluster are automatically updated to include the IP addresses of the Kafka Connect cluster. However, the firewall rules for the Karapace schema registry are not automatically adjusted. It is necessary to manually add the public IP addresses of the Kafka Connect cluster to the Karapace schema registry's allowed addresses. The connection from the Kafka Connect cluster to the schema registry appears to be established over public IPs.

Deploy Kpow instance​

With the Kafka and Kafka Connect clusters running, a Kpow instance can be launched using Docker.

The setup.env file is crucial for establishing the connection between Kpow and our Kafka ecosystem on Instaclustr. Here's a detailed guide on how we can populate this file with the correct information sourced from the Instaclustr console:

  • Kafka Cluster: The bootstrap server addresses, which are the public IP addresses of our Kafka brokers, can be found on the Connection Info page of our Kafka cluster. This page also provides the necessary username and password. These values should be assigned to the BOOTSTRAP and SASL_JAAS_CONFIG variables, respectively.
  • Schema Registry: For the Schema Registry connection, we will need the URL that is secured with a CA-signed certificate. This URL, along with the required username and password, is also available on the Connection Info page for our Schema Registry service.
  • Kafka Connect Cluster: The configuration for our Kafka Connect cluster requires its public IP address for the CONNECT_REST_URL. The credentials (username and password) for the Connect cluster can be found on its respective Connection Info page and should be used in the connection configuration.

💡 To enable a connection from our local machine to the Kafka resources, it is essential to add our IP address to the Firewall Rules within the Instaclustr console for our cluster.

## Kafka environments
ENVIRONMENT_NAME=Instaclustr Demo
BOOTSTRAP=<KAFKA-IP1>:9092,<KAFKA-IP2>:9092,<KAFKA-IP3>:9092
SECURITY_PROTOCOL=SASL_PLAINTEXT
SASL_MECHANISM=SCRAM-SHA-256
SASL_JAAS_CONFIG=org.apache.kafka.common.security.scram.ScramLoginModule required username="<KAFKA_USERNAME>" password="<KAFKA_PASSWORD>";
CONNECT_NAME=Instaclustr Demo Cluster
CONNECT_REST_URL=https://<KAFKA-CONNECT-IP1>:8083
CONNECT_PERMISSIVE_SSL=true
CONNECT_AUTH=BASIC
CONNECT_BASIC_AUTH_USER=<KAFKA_CONNECT_USERNAME>
CONNECT_BASIC_AUTH_PASS=<KAFKA_CONNECT_PASSWORD>
SCHEMA_REGISTRY_NAME=Instaclustr Demo Registry
SCHEMA_REGISTRY_URL=https://<REGISTRY_URL_WITH_AN_ASSOCIATED_CA_SIGNED_CERTIFICATE>:8085
SCHEMA_REGISTRY_AUTH=USER_INFO
SCHEMA_REGISTRY_USER=<SCHEMA_REGISTRY_USERNAME>
SCHEMA_REGISTRY_PASSWORD=<SCHEMA_REGISTRY_PASSWORD>

To launch Kpow, we execute the following Docker command. Ensure that a valid Kpow license file is located at ./kpow/config/license.env.

docker run -d -p 3000:3000 --name kpow \
  --env-file=./kpow/config/setup.env \
  --env-file=./kpow/config/license.env \
  factorhouse/kpow-ce:latest

Once running, Kpow will be accessible at http://<ip-address>:3000. The user interface should confirm the successful discovery of our three Kafka brokers, the Schema Registry, and the Kafka Connect cluster.

Kpow Overview