Audit Logging

Audit logging is supported on BYOC and Dedicated clusters running Redpanda version 24.3 and later. To configure audit logging, see Configure Cluster Properties.

Many scenarios for streaming data include the need for fine-grained auditing of user activity related to the system. This is especially true for regulated industries such as finance, healthcare, and the public sector. Complying with PCI DSS v4 standards, for example, requires verbose and detailed activity auditing, alerting, and analysis capabilities.

Redpanda’s auditing capabilities support recording both administrative and operational interactions with topics and with users. Redpanda complies with the Open Cybersecurity Schema Framework (OCSF), providing a predictable and extensible solution that works seamlessly with industry standard tools.

With audit logging enabled, there should be no noticeable changes in performance other than slightly elevated CPU usage.

Audit log flow

The Redpanda audit log mechanism functions similar to the Kafka flow. When a user interacts with another user or with a topic, Redpanda writes an event to a specialized audit topic. The audit topic is immutable. Only Redpanda can write to it. Users are prevented from writing to the audit topic directly and the Kafka API cannot create or delete it.

Audit log flow

By default, any management and authentication actions performed on the cluster yield messages written to the audit log topic that are retained for seven days. Interactions with all topics by all principals are audited. Actions performed using the Kafka API and Admin API are all audited, as are actions performed directly through rpk.

Messages recorded to the audit log topic comply with the open cybersecurity schema framework. Any number of analytics frameworks, such as Splunk or Sumo Logic, can receive and process these messages. Using an open standard ensures Redpanda’s audit logs coexist with those produced by other IT assets, powering holistic monitoring and analysis of your assets.

Audit log configuration options

Redpanda’s audit logging mechanism supports several options to control the volume and availability of audit records. Configuration is applied at the cluster level.

To configure audit logging, see Configure Cluster Properties.

audit_enabled: Boolean value to enable audit logging. When you set this to true, Redpanda checks for an existing topic named _redpanda.audit_log. If none is found, Redpanda automatically creates one for you. Default: true.
audit_enabled_event_types: List of strings in JSON style identifying the event types to include in the audit log. This may include any of the following: management, produce, consume, describe, heartbeat, authenticate, schema_registry, admin. Default: '["management","authenticate","admin"]'.
audit_excluded_principals: List of strings in JSON style identifying the principals the audit logging system should ignore. Principals can be listed as User:name or name, both are accepted. Default: null.

Enable audit logging

Audit logging is enabled by default. Cluster administrators can configure the audited topics and principals. However, only the Redpanda team can configure the type of audited events. For more information or support, contact your Redpanda account team.

Configure retention for audit logs

You can export audit events to your SIEM for long-term retention to support audit and compliance needs. Redpanda Data recommends that you retain audit logs for at least one year in a separate system like your SIEM, so if there is an issue with the Redpanda cluster you have access to the audit logs.

If you need to change the default seven-day retention period, update the retention settings using the retention.ms property for the _redpanda.audit_log topic:

# Set 1-year retention (in milliseconds) on the audit log topic
rpk topic alter-config _redpanda.audit_log --set retention.ms=31536000000

In Redpanda Cloud, both retention.ms (time-based) and retention.bytes (size-based) retention policies are applied simultaneously. Data becomes eligible for deletion when either limit is reached, depending on whichever occurs first. This means neither setting strictly takes precedence; the earliest limit (by time or size) triggers data cleanup. When updating audit log retention, check to make sure you do not already have a size-based retention policy that might remove logs before the period you specify.