Docs Cloud Develop Managed Connectors Sizing Connectors Sizing Connectors Connector tasks When you set up a connector, its main responsibility is to validate the configuration and spawn connector tasks, which perform the work. Setting up multiple tasks for a connector allows for parallelization of the work, resulting in higher throughputs. Before setting up connector tasks, consider the following: For source connectors, the ability to add tasks to achieve higher throughput depends on the connector implementation and configuration. For many connectors, only a single connector task is allowed (for example, Debezium allows a single task only). When Redpanda Cloud does not offer an option to set the number of tasks, the source connector runs only one task. For sink connectors, parallelism is achieved by evenly distributing configured topic partitions for the connector amongst connector tasks. The number of partitions must be equal to or greater than the number of tasks. Single task throughput Connector throughput depends on many factors, including converters used, compression, message size, and the performance of external systems. As a rule of thumb, expect a single connector task to provide 1-2 MB/s of throughput. Specify number of connector tasks for a sink connector It can be a challenge to determine the number of connector tasks to use for a given workload, so you must experiment to find the right number. Start with low number of connector tasks and wait a couple of minutes to view performance. Keep increasing the number of tasks until satisfactory throughput is achieved. Keep in mind that the underlying infrastructure must scale to provide room for additional connector tasks. Waiting roughly 10 minutes after each change should provide sufficient time for the system to scale up. Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution Single Message Transforms AWS S3 Sink Connector