Docs Self-Managed Troubleshoot Generate Debug Bundle Redpanda Console You are viewing the Self-Managed v24.3 beta documentation. We welcome your feedback at the Redpanda Community Slack #beta-feedback channel. To view the latest available version of the docs, see v24.2. Manage Debug Bundles in Redpanda Console This section pertains to Redpanda Console in a self-managed deployment, and this feature requires an Enterprise license. To upgrade, contact Redpanda sales. If Redpanda Console has enterprise features enabled and it cannot find a valid license either in its local configuration or in the connected Redpanda cluster, it shuts down. Learn how to generate, download, and delete debug bundles in Redpanda Console for comprehensive cluster diagnostics. Prerequisites Redpanda Console must be connected to a Redpanda cluster and configured to connect to the Redpanda Admin API. Redpanda Console must be configured with an authentication provider and have at least one admin user. Generate a debug bundle You can generate a debug bundle for all brokers in the cluster using Redpanda Console. After generating the bundle, download it onto your local computer for inspection. Log in as an admin user. Click Admin in the sidebar. This option is visible only when you are logged in as an admin user. Go to Debug bundle. Click Generate new. Wait until the process is complete. Click debug-bundle.zip to download the bundle on your local computer. Unzip the file to inspect the contents. Inspect the debug bundle After downloading the debug bundle files, you can inspect the contents to debug your cluster. This section provides some useful data points to check while troubleshooting. Most files in the debug bundle are JSON files. To make it easier to read these files, this section uses jq. To install jq, see the jq downloads page. View the version of Redpanda on all brokers cat admin/brokers.json | jq '.[] | .version' Example output: "v24.3.1" "v24.3.1" "v24.3.1" View the maintenance status of all brokers cat admin/brokers.json | jq '.[] | .node_id, .maintenance_status' Example output 0 { "draining": false, "finished": false, "errors": false, "partitions": 0, "eligible": 0, "transferring": 0, "failed": 0 } 1 { "draining": false, "finished": false, "errors": false, "partitions": 0, "eligible": 0, "transferring": 0, "failed": 0 } 2 { "draining": false, "finished": false, "errors": false, "partitions": 0, "eligible": 0, "transferring": 0, "failed": 0 } View the cluster configuration cat admin/cluster_config.json | jq Example output { "abort_index_segment_size": 50000, "abort_timed_out_transactions_interval_ms": 10000, "admin_api_require_auth": false, "aggregate_metrics": false, "alter_topic_cfg_timeout_ms": 5000, "append_chunk_size": 16384, "auto_create_topics_enabled": false, "cloud_storage_access_key": null, "cloud_storage_api_endpoint": null, "cloud_storage_api_endpoint_port": 443, "cloud_storage_azure_container": null, "cloud_storage_azure_shared_key": null, "cloud_storage_azure_storage_account": null, "cloud_storage_bucket": null, ... "target_quota_byte_rate": 2147483648, "tm_sync_timeout_ms": 10000, "topic_fds_per_partition": 5, "topic_memory_per_partition": 1048576, "topic_partitions_per_shard": 1000, "topic_partitions_reserve_shard0": 2, "transaction_coordinator_cleanup_policy": "delete", "transaction_coordinator_delete_retention_ms": 604800000, "transaction_coordinator_log_segment_size": 1073741824, "transactional_id_expiration_ms": 604800000, "tx_log_stats_interval_s": 10, "tx_timeout_delay_ms": 1000, "wait_for_leader_timeout_ms": 5000, "zstd_decompress_workspace_bytes": 8388608 } Check Enterprise Edition license keys cat admin/license.json | jq Example output { "loaded": false, "license": { "format_version": 0, "org": "", "type": "", "expires": 0, "sha256": "" } } View metadata about the Redpanda data directory To check the size of the directories and look for anomalies: cat utils/du.txt Example output 33M /var/lib/redpanda/data/redpanda/kvstore/0_0 33M /var/lib/redpanda/data/redpanda/kvstore 33M /var/lib/redpanda/data/redpanda/controller/0_0 33M /var/lib/redpanda/data/redpanda/controller 65M /var/lib/redpanda/data/redpanda 65M /var/lib/redpanda/data To check the file permissions, file size, and last modification date of the files: cat data-dir.txt | jq Example output { "/var/lib/redpanda/data": { "size": "4.096kB", "mode": "dgrwxrwxrwx", "modified": "2023-02-02 15:21:12.430878371 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/config_cache.yaml": { "size": "340B", "mode": "-rw-r--r--", "modified": "2023-02-02 15:21:22.434878593 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/pid.lock": { "size": "2B", "mode": "-rw-r--r--", "modified": "2023-02-02 15:21:10.502878322 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/redpanda": { "size": "4.096kB", "mode": "dgrwxr-xr-x", "modified": "2023-02-02 15:21:10.650878326 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/redpanda/controller": { "size": "4.096kB", "mode": "dgrwxr-xr-x", "modified": "2023-02-02 15:21:10.650878326 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/redpanda/controller/0_0": { "size": "4.096kB", "mode": "dgrwxr-xr-x", "modified": "2023-02-02 15:21:12.346878368 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/redpanda/controller/0_0/0-1-v1.log": { "size": "4.096kB", "mode": "-rw-r--r--", "modified": "2023-02-02 15:21:32.450878771 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/redpanda/kvstore": { "size": "4.096kB", "mode": "dgrwxr-xr-x", "modified": "2023-02-02 15:21:10.590878324 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/redpanda/kvstore/0_0": { "size": "4.096kB", "mode": "dgrwxr-xr-x", "modified": "2023-02-02 15:21:10.602878325 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/redpanda/kvstore/0_0/0-0-v1.log": { "size": "8.192kB", "mode": "-rw-r--r--", "modified": "2023-02-02 15:21:32.458878772 +0000 UTC", "user": "", "group": "redpanda" }, "/var/lib/redpanda/data/startup_log": { "size": "26B", "mode": "-rw-r--r--", "modified": "2023-02-02 15:21:10.510878323 +0000 UTC", "user": "", "group": "redpanda" } } View cluster metadata cat kafka.json | jq '.[0]' Example output { "Name": "metadata", "Response": { "Cluster": "redpanda.14a3f9b6-1c74-4ffd-806a-4ab48db78120", "Controller": 0, "Brokers": [ { "NodeID": 0, "Port": 9093, "Host": "redpanda-0.redpanda.<namespace>.svc.cluster.local.", "Rack": null }, { "NodeID": 1, "Port": 9093, "Host": "redpanda-1.redpanda.<namespace>.svc.cluster.local.", "Rack": null }, { "NodeID": 2, "Port": 9093, "Host": "redpanda-2.redpanda.<namespace>.svc.cluster.local.", "Rack": null } ], "Topics": {} }, "Error": null } View topic and broker configurations cat kafka.json | jq '.[1:]' Example output [ { "Name": "topic_configs", "Response": null, "Error": null }, { "Name": "broker_configs", "Response": [ { "Name": "0", "Configs": [ { "Key": "listeners", "Value": "internal://0.0.0.0:9093,default://0.0.0.0:9094", "Sensitive": false, "Source": "STATIC_BROKER_CONFIG", "Synonyms": [ { "Key": "kafka_api", "Value": "internal://0.0.0.0:9093,default://0.0.0.0:9094", "Source": "STATIC_BROKER_CONFIG" }, { "Key": "kafka_api", "Value": "plain://127.0.0.1:9092", "Source": "DEFAULT_CONFIG" } ] }, { "Key": "advertised.listeners", "Value": "internal://redpanda-0.redpanda.<namespace>.svc.cluster.local.:9093,default://203.0.113.3:31092", "Sensitive": false, "Source": "STATIC_BROKER_CONFIG", "Synonyms": [ { "Key": "advertised_kafka_api", "Value": "internal://redpanda-0.redpanda.<namespace>.svc.cluster.local.:9093,default://203.0.113.3:31092", "Source": "STATIC_BROKER_CONFIG" }, { "Key": "advertised_kafka_api", "Value": "", "Source": "DEFAULT_CONFIG" } ] }, { "Key": "log.segment.bytes", "Value": "134217728", "Sensitive": false, "Source": "DEFAULT_CONFIG", "Synonyms": [ { "Key": "log_segment_size", "Value": "134217728", "Source": "DEFAULT_CONFIG" } ] }, { "Key": "log.retention.bytes", "Value": "18446744073709551615", "Sensitive": false, "Source": "DEFAULT_CONFIG", "Synonyms": [ { "Key": "retention_bytes", "Value": "18446744073709551615", "Source": "DEFAULT_CONFIG" } ] }, { "Key": "log.retention.ms", "Value": "604800000", "Sensitive": false, "Source": "DEFAULT_CONFIG", "Synonyms": [ { "Key": "delete_retention_ms", "Value": "604800000", "Source": "DEFAULT_CONFIG" } ] }, { "Key": "num.partitions", "Value": "1", "Sensitive": false, "Source": "DEFAULT_CONFIG", "Synonyms": [ { "Key": "default_topic_partitions", "Value": "1", "Source": "DEFAULT_CONFIG" } ] }, { "Key": "default.replication.factor", "Value": "1", "Sensitive": false, "Source": "DEFAULT_CONFIG", "Synonyms": [ { "Key": "default_topic_replications", "Value": "1", "Source": "DEFAULT_CONFIG" } ] }, { "Key": "log.dirs", "Value": "/var/lib/redpanda/data", "Sensitive": false, "Source": "STATIC_BROKER_CONFIG", "Synonyms": [ { "Key": "data_directory", "Value": "/var/lib/redpanda/data", "Source": "STATIC_BROKER_CONFIG" } ] }, { "Key": "auto.create.topics.enable", "Value": "false", "Sensitive": false, "Source": "DEFAULT_CONFIG", "Synonyms": [ { "Key": "auto_create_topics_enabled", "Value": "false", "Source": "DEFAULT_CONFIG" } ] } ], "Err": null }, { "Name": "1", "Configs": [ ... ] ... }, { "Name": "1", "Configs": [ ... ] ... }, ], "Error": null }, { "Name": "log_start_offsets", "Response": {}, "Error": null }, { "Name": "last_stable_offsets", "Response": {}, "Error": null }, { "Name": "high_watermarks", "Response": {}, "Error": null }, { "Name": "groups", "Response": null, "Error": null } ] View the Redpanda logs cat redpanda.log Check for clock drift cat utils/ntp.txt | jq Use the output to check for clock drift. For details about how NTP works, see the NTP documentation. Example output { "host": "pool.ntp.org", "roundTripTimeMs": 3, "remoteTimeUTC": "2023-02-02T15:22:51.763175934Z", "localTimeUTC": "2023-02-02T15:22:51.698044603Z", "precisionMs": 0, "offset": -458273 } Delete a debug bundle To generate a new debug bundle, you must delete the existing one. Log in as an admin user. Click Admin in the sidebar. This option is visible only when you are logged in as an admin user. Go to Debug bundle. Click the trash icon next to debug-bundle.zip to delete the bundle. Next steps Cluster Diagnostics Error Messages and Solutions Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Kubernetes Resolve Errors