Docs Self-Managed Get Started What’s New This is documentation for Self-Managed v24.1. To view the latest available version of the docs, see v24.2. What’s New This topic includes new content added in version 24.1. For a complete list of all product updates, see the Redpanda release notes. See also: See also: What’s New in Redpanda Cloud Redpanda Connect In release 24.1.5, Redpanda announced Redpanda Connect. This includes over 220 open source connectors, with a large collection of sources, sinks, and processors. The interface provides a lightweight and easy way to set up, debug, and manage connectors. Write caching Write caching is a relaxed mode of acks=all that provides better performance at the expense of durability. It acknowledges a message as soon as it is received and acknowledged on a majority of brokers, without waiting for it to fsync to disk. This provides lower latency while still ensuring that a majority of brokers acknowledge the write. For clusters in development mode, write caching is enabled by default. For clusters in production mode, it is disabled by default. RBAC Role-based access control (RBAC) provides an extension to Redpanda’s existing access control lists (ACLs). Using rpk or Redpanda Console, you can create roles and assign those roles as the authorized or denied party for an ACL. Every user you assign to that role inherits those privileges. RBAC provides a more flexible and efficient way to manage user privileges, especially with complex organizational hierarchies or large numbers of users. Data transforms Data transforms is now generally available for all Redpanda Community and Redpanda Enterprise users. This provides the framework to build and deploy inline transformations on data written to Redpanda topics using WebAssembly, delivering processed and validated data to consumers in the format they expect. The following features have been added since the 23.3 beta release: Data transforms can now output to multiple topics. For example, you can distribute different message types to multiple event streams, or you can write valid records to an output topic and invalid records to a dead-letter queue. There is a new Rust SDK for data transforms. This complements the existing Golang SDK. Data transform logs are now stored in an internally-managed topic and are accessible via rpk and Redpanda Console. Enhanced cache trimming From 24.1.10, Redpanda has two new properties that provide finer control over cache management. These settings allow you to define specific thresholds for triggering cache trimming based on cache size and the number of objects, helping to optimize performance and prevent slow reads. cloud_storage_cache_trim_threshold_percent_size cloud_storage_cache_trim_threshold_percent_objects Topic-aware partition balancing Redpanda now prioritizes balancing a topic’s partition replica count evenly across all brokers while it’s balancing the cluster’s overall partition count. Because different topics in a cluster can have vastly different load profiles, this better distributes the workload of the most heavily-used topics evenly across brokers. This feature is enabled by default with the partition_autobalancing_topic_aware property. After upgrading to 24.1, new topics are distributed in a topic-aware fashion, and the next partition rebalance (triggered by Continuous Data Balancing, by adding new brokers, or by calling the /v1/partitions/rebalance Admin API endpoint) evens out the partition counts on each broker. Managed identities for Azure Redpanda Tiered Storage now supports managed identities for Azure resources, similar to the existing support for IAM roles for self-hosted deployments on AWS and GCP. This provides a more secure alternative to static credentials, like access keys. Managed identities allow Redpanda clusters deployed on Azure virtual machines and on Azure Kubernetes Service (AKS) to access Azure Blob Storage (ABS) and other services without storing credentials in the cluster configuration. Improved fetch latency A new property, fetch_read_strategy, by default sets new and migrated clusters with a non-polling strategy to fulfill fetch requests. The backend is signaled when a partition has new data, so it doesn’t need to repeatedly read from every partition in the fetch. This should improve fetch latency and CPU utilization for most workloads. To revert to the previous polling strategy, run: rpk cluster config set fetch_read_strategy polling Validation before topic recovery or cluster restore Before topic recovery or cluster restore, the properties cloud_storage_recovery_topic_validation_mode and cloud_storage_recovery_topic_validation_depth now check that partition manifests exist and confirm that there are no connection issues to object storage. New commands The following rpk commands are new in this release: rpk cluster partitions transfer-leadership rpk redpanda tune help rpk transform logs rpk connect commands rpk security commands Additionally, to bundle security commands together, the existing rpk acl commands have moved into the new rpk security section as rpk security acl. The existing rpk acl commands still work as an alias. Documentation enhancements Redpanda Labs provides examples, guidance, best practices, and ideas for how to use Redpanda in your own project. Redpanda Connect documentation is added as a new dropdown option, similar to Redpanda Labs. Improved search lets you filter for blogs, videos, labs, and documentation for all supported versions. New side-by-side codeblocks let you easily compare code examples. Improved listing of changes to rpk commands in deprecated features. List of Redpanda integrations built and supported by our partners. Next steps Upgrade Redpanda Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Get Started Introduction to Redpanda