# Internal Metrics Reference

> For the complete documentation index, see [llms.txt](https://docs.redpanda.com/llms.txt). Component-specific: [streaming-full.txt](https://docs.redpanda.com/streaming-full.txt)

---
title: Internal Metrics Reference
latest-operator-version: v26.1.4
# EOL = End-of-Life (support lifecycle status)
page-is-nearing-eol: "false"
page-is-past-eol: "true"
page-eol-date: December 22, 2024
latest-console-tag: v3.7.3
latest-connect-version: 4.93.0
docname: internal-metrics-reference
page-component-name: streaming
page-version: "23.3"
page-component-version: "23.3"
page-component-title: Streaming
page-relative-src-path: internal-metrics-reference.adoc
page-edit-url: https://github.com/redpanda-data/docs/edit/v/23.3/modules/reference/pages/internal-metrics-reference.adoc
description: Reference of Redpanda internal metrics for detailed analysis, debugging, and troubleshooting.
page-git-created-date: "2023-05-30"
page-git-modified-date: "2024-01-09"
support-status: past end-of-life
---

<!-- Source: https://docs.redpanda.com/streaming/23.3/reference/internal-metrics-reference.md -->

This section provides reference descriptions about the internal metrics exported from Redpanda’s `/metrics` endpoint.

> 💡 **TIP**
>
> Use [/public\_metrics](https://docs.redpanda.com/streaming/23.3/reference/public-metrics-reference/) for your primary dashboards for system health.
>
> Use [/metrics](./) for detailed analysis and debugging.

> ❗ **IMPORTANT**
>
> In a live system, Redpanda metrics are exported only for features that are in use. For example, a metric for consumer groups is not exported when no groups are registered.
>
> To see the available internal metrics in your system, query the `/metrics` endpoint:
>
> ```bash
> curl http://<node-addr>:9644/metrics | grep "[HELP|TYPE]"
> ```

## [](#internal-metrics)Internal metrics

Most internal metrics are useful for debugging. The following subset of internal metrics can be useful to monitor system health.

* * *

### [](#vectorized_application_uptime)vectorized_application_uptime

Redpanda uptime in milliseconds.

* * *

### [](#vectorized_cluster_partition_last_stable_offset)vectorized_cluster_partition_last_stable_offset

Last stable offset.

If this is the last record received by the cluster, then the cluster is up-to-date and ready for maintenance.

* * *

### [](#vectorized_cluster_partition_schema_id_validation_records_failed)vectorized_cluster_partition_schema_id_validation_records_failed

Number of records that failed schema ID validation.

* * *

### [](#vectorized_cluster_partition_start_offset)vectorized_cluster_partition_start_offset

Raft snapshot start offset.

* * *

### [](#vectorized_io_queue_delay)vectorized_io_queue_delay

Total delay time in the queue.

Can indicate latency caused by disk operations in seconds.

* * *

### [](#vectorized_io_queue_queue_length)vectorized_io_queue_queue_length

Number of requests in the queue.

Can indicate latency caused by disk operations.

* * *

### [](#vectorized_kafka_rpc_active_connections)vectorized_kafka_rpc_active_connections

Number of currently active Kafka RPC connections, or clients.

* * *

### [](#vectorized_kafka_rpc_connects)vectorized_kafka_rpc_connects

Number of accepted Kafka RPC connections.

Compare to the value at a previous time to derive the rate of accepted connections.

* * *

### [](#vectorized_kafka_rpc_produce_bad_create_time)vectorized_kafka_rpc_produce_bad_create_time

An incrementing counter for the number of times a producer created a message with a timestamp skewed from the broker’s date and time. This metric is related to the following properties:

-   `log_message_timestamp_alert_before_ms`: Increment this gauge when the `create_timestamp` on a message is too far in the past as compared to the broker’s time.

-   `log_message_timestamp_alert_after_ms`: Increment this gauge when the `create_timestamp` on a message is too far in the future as compared to the broker’s time.


* * *

### [](#vectorized_kafka_rpc_received_bytes)vectorized_kafka_rpc_received_bytes

Number of bytes received from Kafka RPC clients in valid requests.

Compare to the value at a previous time to derive the throughput in Kafka layer in bytes/sec received.

* * *

### [](#vectorized_kafka_rpc_requests_completed)vectorized_kafka_rpc_requests_completed

Number of successful Kafka RPC requests.

Compare to the value at a previous time to derive the messages per second per shard.

* * *

### [](#vectorized_kafka_rpc_requests_pending)vectorized_kafka_rpc_requests_pending

Number of Kafka RPC requests being processed by a server.

* * *

### [](#vectorized_kafka_rpc_sent_bytes)vectorized_kafka_rpc_sent_bytes

Number of bytes sent to Kafka RPC clients.

* * *

### [](#vectorized_kafka_rpc_service_errors)vectorized_kafka_rpc_service_errors

Number of Kafka RPC service errors.

* * *

### [](#vectorized_raft_leadership_changes)vectorized_raft_leadership_changes

Number of leadership changes.

High value can indicate nodes failing and causing leadership changes.

* * *

### [](#vectorized_reactor_utilization)vectorized_reactor_utilization

Redpanda process utilization.

Shows the true utilization of the CPU by a Redpanda process.

* * *

### [](#vectorized_storage_log_compacted_segment)vectorized_storage_log_compacted_segment

Number of compacted segments.

* * *

### [](#vectorized_storage_log_log_segments_created)vectorized_storage_log_log_segments_created

Number of created log segments.

* * *

### [](#vectorized_storage_log_partition_size)vectorized_storage_log_partition_size

Current size of partition in bytes.

* * *

### [](#vectorized_storage_log_read_bytes)vectorized_storage_log_read_bytes

Total number of bytes read.

* * *

### [](#vectorized_storage_log_written_bytes)vectorized_storage_log_written_bytes

Total number of bytes written.

## [](#related-topics)Related topics

-   [Learn how to monitor Redpanda](https://docs.redpanda.com/streaming/23.3/manage/monitoring/)

-   [Public metrics reference](https://docs.redpanda.com/streaming/23.3/reference/public-metrics-reference/)