Streaming

Connect Redpanda Console to Kafka Connect Clusters

Kafka Connect is community-supported on Redpanda Community Slack. Redpanda Data does not provide enterprise support for Kafka Connect with Redpanda Console. For a supported and scalable Kafka Connect alternative, try Redpanda Connect.

Redpanda Console provides a user interface that lets you manage multiple Kafka Connect clusters. You can inspect or patch connectors; restart, pause, and resume connector tasks; and delete connectors. Redpanda Console queries all configured Kafka Connect clusters for their status, so you have an overview of all your Kafka Connect clusters.

Prerequisites

You must deploy a Kafka Connect cluster separately before configuring Redpanda Console to connect to it.

Configure a connection to Kafka Connect

For each cluster, provide a unique name, the HTTP address of the cluster, and the authentication settings, if required. The name can be any unique string that helps you to identify the Kafka Connect cluster. See all available configuration options in the Redpanda Console Configuration.

  • Standalone

  • Kubernetes embedded

  • Kubernetes standalone

kafkaConnect:
  enabled: true
  clusters:
    - name: datawarehouse # Required field, will be used as identifier in the frontend
      url: http://dwh-connect.mycompany.com:8083
      tls:
        enabled: false # Trusted certs are still allowed by default
      username: admin
      # password: # Set using flag --kafkaConnect.clusters.0.password=secret
    - name: analytics # Required field, will be used as identifier in the frontend
      url: http://analytics.mycompany.com:8083
      # No auth configured on that cluster, hence no username/password set

When Redpanda Console is part of the Redpanda Helm chart or Operator:

  • Operator

  • Helm

redpanda-console.yaml
apiVersion: cluster.redpanda.com/v1alpha2
kind: Console
metadata:
  name: redpanda-console
spec:
  clusterRef:
    name: redpanda
  config:
    kafkaConnect:
      enabled: true
      clusters:
        - name: datawarehouse
          url: http://dwh-connect.mycompany.com:8083
          tls:
            enabled: false
          username: admin
          # password: (add if needed)
        - name: analytics
          url: http://analytics.mycompany.com:8083
          # No auth configured on this cluster
redpanda-values.yaml
console:
  enabled: true
  console:
    config:
      kafkaConnect:
        enabled: true
        clusters:
          - name: datawarehouse # Required field, will be used as identifier in the frontend
            url: http://dwh-connect.mycompany.com:8083
            tls:
              enabled: false # Trusted certs are still allowed by default
            username: admin
            # password:
          - name: analytics # Required field, will be used as identifier in the frontend
            url: http://analytics.mycompany.com:8083
            # No auth configured on that cluster, hence no username/password set

When using the standalone Redpanda Console Helm chart:

console-values.yaml
config:
  kafkaConnect:
    enabled: true
    clusters:
      - name: datawarehouse # Required field, will be used as identifier in the frontend
        url: http://dwh-connect.mycompany.com:8083
        tls:
          enabled: false # Trusted certs are still allowed by default
        username: admin
        # password:
      - name: analytics # Required field, will be used as identifier in the frontend
        url: http://analytics.mycompany.com:8083
        # No auth configured on that cluster, hence no username/password set