Cloud

Monitor Kafka Connect

  • To enable this feature, contact Redpanda Support. To disable this feature, see Disable Kafka Connect.

  • Redpanda Support does not manage or monitor Kafka Connect. For fully-supported connectors, consider Redpanda Connect.

  • When Kafka Connect is enabled, there is a dedicated node running even when no connectors are deployed.

You can monitor the health of Kafka Connect with metrics that Redpanda exports through a Prometheus HTTPS endpoint. You can use Grafana to visualize the metrics and set up alerts.

The most important metrics to be monitored by alerts are:

  • connector failed tasks

  • connector lag / connector lag rate

View connector logs

Connector logs are written to the system topic __redpanda.connectors_logs. You can view logs in Redpanda Cloud on the Topics page for your cluster, or you can download logs with rpk. For example:

# Last 100 messages (most recent)
rpk topic consume __redpanda.connectors_logs -o -100 -n 100

# Last 10 minutes
rpk topic consume __redpanda.connectors_logs -o @-10m:end

# Stream new logs only (like tail -f)
rpk topic consume __redpanda.connectors_logs -o end

# Filter by connector name
rpk topic consume __redpanda.connectors_logs -o @-10m:end -O json \
  | jq -r 'select(.message | test("<connector-name>"; "i"))'
Access to system topics may be restricted by organization/project roles. Log retention follows cluster/system-topic policies and messages may expire.

Limitations

The connectors dashboard renders metrics that are exported by managed connectors. However, when a connector does not create a task (for example, an empty topic list), the dashboard will not show metrics for that connector.