# Connect: components/processors

> Index of Connect components/processors documentation pages. Each link points to the page's markdown version.
> For the full documentation index, see https://docs.redpanda.com/llms.txt

## Pages

- [Processors](https://docs.redpanda.com/connect/components/processors/about.md): Learn how processors work in Redpanda Connect. Processors are functions that mutate, filter, or enrich messages passing through a pipeline.
- [archive](https://docs.redpanda.com/connect/components/processors/archive.md): Archives all the messages of a batch into a single message according to the selected archive format.
- [avro](https://docs.redpanda.com/connect/components/processors/avro.md): Performs Avro based operations on messages based on a schema.
- [awk](https://docs.redpanda.com/connect/components/processors/awk.md): Executes an AWK program on messages. This processor is very powerful as it offers a range of custom functions for querying and mutating message contents and metadata.
- [aws_bedrock_chat](https://docs.redpanda.com/connect/components/processors/aws_bedrock_chat.md): Generates responses to messages in a chat conversation, using the AWS Bedrock API.
- [aws_bedrock_embeddings](https://docs.redpanda.com/connect/components/processors/aws_bedrock_embeddings.md): Computes vector embeddings on text, using the AWS Bedrock API.
- [aws_dynamodb_partiql](https://docs.redpanda.com/connect/components/processors/aws_dynamodb_partiql.md): Executes a PartiQL expression against a DynamoDB table for each message.
- [aws_lambda](https://docs.redpanda.com/connect/components/processors/aws_lambda.md): Invokes an AWS lambda for each message. The contents of the message is the payload of the request, and the result of the invocation will become the new contents of the message.
- [azure_cosmosdb](https://docs.redpanda.com/connect/components/processors/azure_cosmosdb.md): Creates or updates messages as JSON documents in Azure CosmosDB.
- [benchmark](https://docs.redpanda.com/connect/components/processors/benchmark.md): Logs basic throughput statistics of messages that pass through this processor.
- [bloblang](https://docs.redpanda.com/connect/components/processors/bloblang.md): Executes a Bloblang mapping on messages.
- [bounds_check](https://docs.redpanda.com/connect/components/processors/bounds_check.md): Removes messages (and batches) that do not fit within certain size boundaries.
- [branch](https://docs.redpanda.com/connect/components/processors/branch.md): The branch processor allows you to create a new request message via a Bloblang mapping, execute a list of processors on the request messages, and, finally, map the result back into the source message...
- [cache](https://docs.redpanda.com/connect/components/processors/cache.md): Performs operations against a cache resource for each message, allowing you to store or retrieve data within message payloads.
- [cached](https://docs.redpanda.com/connect/components/processors/cached.md): Cache the result of applying one or more processors to messages identified by a key.
- [catch](https://docs.redpanda.com/connect/components/processors/catch.md): Applies a list of child processors _only_ when a previous processing step has failed.
- [cohere_chat](https://docs.redpanda.com/connect/components/processors/cohere_chat.md): Generates responses to messages in a chat conversation, using the Cohere API.
- [cohere_embeddings](https://docs.redpanda.com/connect/components/processors/cohere_embeddings.md): Generates vector embeddings to represent input text, using the Cohere API.
- [cohere_rerank](https://docs.redpanda.com/connect/components/processors/cohere_rerank.md): Generates vector embeddings to represent input text, using the Cohere API.
- [command](https://docs.redpanda.com/connect/components/processors/command.md): Executes a command for each message.
- [compress](https://docs.redpanda.com/connect/components/processors/compress.md): Compresses messages according to the selected algorithm. Supported compression algorithms are: [flate gzip lz4 pgzip snappy zlib].
- [couchbase](https://docs.redpanda.com/connect/components/processors/couchbase.md): Performs operations against Couchbase for each message, allowing you to store or retrieve data within message payloads.
- [crash](https://docs.redpanda.com/connect/components/processors/crash.md): Crashes the process using a fatal log message. The log message can be set using function interpolations described in Bloblang queries which allows you to log the contents and metadata of messages.
- [decompress](https://docs.redpanda.com/connect/components/processors/decompress.md): Decompresses messages according to the selected algorithm. Supported decompression algorithms are: [bzip2 flate gzip lz4 pgzip snappy zlib].
- [dedupe](https://docs.redpanda.com/connect/components/processors/dedupe.md): Deduplicates messages by storing a key value in a cache using the add operator. If the key already exists within the cache it is dropped.
- [ffi](https://docs.redpanda.com/connect/components/processors/ffi.md): Invoke a function within a shared library as a processing step.
- [for_each](https://docs.redpanda.com/connect/components/processors/for_each.md): A processor that applies a list of child processors to messages of a batch as though they were each a batch of one message.
- [gcp_bigquery_select](https://docs.redpanda.com/connect/components/processors/gcp_bigquery_select.md): Executes a SELECT query against BigQuery and replaces messages with the rows returned.
- [gcp_vertex_ai_chat](https://docs.redpanda.com/connect/components/processors/gcp_vertex_ai_chat.md): Generates responses to messages in a chat conversation, using the Vertex AI API.
- [gcp_vertex_ai_embeddings](https://docs.redpanda.com/connect/components/processors/gcp_vertex_ai_embeddings.md): Generates vector embeddings to represent input text, using the Vertex AI API.
- [google_drive_download](https://docs.redpanda.com/connect/components/processors/google_drive_download.md): Downloads files from Google Drive.
- [google_drive_list_labels](https://docs.redpanda.com/connect/components/processors/google_drive_list_labels.md): Lists labels for a file in Google Drive.
- [google_drive_search](https://docs.redpanda.com/connect/components/processors/google_drive_search.md): Searches Google Drive for files matching the provided query.
- [grok](https://docs.redpanda.com/connect/components/processors/grok.md): Parses messages into a structured format by attempting to apply a list of Grok expressions, the first expression to result in at least one value replaces the original message with a JSON object...
- [group_by_value](https://docs.redpanda.com/connect/components/processors/group_by_value.md): Splits a batch of messages into N batches, where each resulting batch contains a group of messages determined by a function interpolated string evaluated per message.
- [group_by](https://docs.redpanda.com/connect/components/processors/group_by.md): Splits a batch of messages into N batches, where each resulting batch contains a group of messages determined by a Bloblang query.
- [http](https://docs.redpanda.com/connect/components/processors/http.md): Performs an HTTP request using a message batch as the request body, and replaces the original message parts with the body of the response.
- [insert_part](https://docs.redpanda.com/connect/components/processors/insert_part.md): Insert a new message into a batch at an index. If the specified index is greater than the length of the existing batch it will be appended to the end.
- [javascript](https://docs.redpanda.com/connect/components/processors/javascript.md): Executes a provided JavaScript code block or file for each message.
- [jira](https://docs.redpanda.com/connect/components/processors/jira.md): Queries Jira resources and returns structured data.
- [jmespath](https://docs.redpanda.com/connect/components/processors/jmespath.md): Executes a JMESPath query on JSON documents and replaces the message with the resulting document.
- [jq](https://docs.redpanda.com/connect/components/processors/jq.md): Transforms and filters messages using jq queries.
- [json_schema](https://docs.redpanda.com/connect/components/processors/json_schema.md): Checks messages against a provided JSONSchema definition but does not change the payload under any circumstances. If a message does not match the schema it can be caught using error handling methods.
- [log](https://docs.redpanda.com/connect/components/processors/log.md): Prints a log event for each message. Messages always remain unchanged. The log message can be set using function interpolations described in Bloblang queries which allows you to log the contents and...
- [mapping](https://docs.redpanda.com/connect/components/processors/mapping.md): Executes a Bloblang mapping on messages, creating a new document that replaces (or filters) the original message.
- [metric](https://docs.redpanda.com/connect/components/processors/metric.md): Emit custom metrics by extracting values from messages.
- [mongodb](https://docs.redpanda.com/connect/components/processors/mongodb.md): Performs operations against MongoDB for each message, allowing you to store or retrieve data within message payloads.
- [msgpack](https://docs.redpanda.com/connect/components/processors/msgpack.md): Converts messages to or from the MessagePack format.
- [mutation](https://docs.redpanda.com/connect/components/processors/mutation.md): Executes a Bloblang mapping and directly transforms the contents of messages, mutating (or deleting) them.
- [nats_kv](https://docs.redpanda.com/connect/components/processors/nats_kv.md): Perform operations on a NATS key-value bucket.
- [nats_request_reply](https://docs.redpanda.com/connect/components/processors/nats_request_reply.md): Sends a message to a NATS subject and expects a reply, from a NATS subscriber acting as a responder, back.
- [noop](https://docs.redpanda.com/connect/components/processors/noop.md): Noop is a processor that does nothing, the message passes through unchanged. Why? Sometimes doing nothing is the braver option.
- [ollama_chat](https://docs.redpanda.com/connect/components/processors/ollama_chat.md): Generates responses to messages in a chat conversation, using the Ollama API.
- [ollama_embeddings](https://docs.redpanda.com/connect/components/processors/ollama_embeddings.md): Generates vector embeddings from text, using the Ollama API.
- [ollama_moderation](https://docs.redpanda.com/connect/components/processors/ollama_moderation.md): Checks whether responses from a large language model (LLM) are safe, using the Ollama API.
- [openai_chat_completion](https://docs.redpanda.com/connect/components/processors/openai_chat_completion.md): Generates responses to messages in a chat conversation, using the OpenAI API.
- [openai_embeddings](https://docs.redpanda.com/connect/components/processors/openai_embeddings.md): Generates vector embeddings to represent input text, using the OpenAI API.
- [openai_image_generation](https://docs.redpanda.com/connect/components/processors/openai_image_generation.md): Generates an image from a text description and other attributes, using OpenAI API.
- [openai_speech](https://docs.redpanda.com/connect/components/processors/openai_speech.md): Generates audio from a text description and other attributes, using OpenAI API.
- [openai_transcription](https://docs.redpanda.com/connect/components/processors/openai_transcription.md): Generates a transcription of spoken audio in the input language, using the OpenAI API.
- [openai_translation](https://docs.redpanda.com/connect/components/processors/openai_translation.md): Translates spoken audio into English, using the OpenAI API.
- [parallel](https://docs.redpanda.com/connect/components/processors/parallel.md): A processor that applies a list of child processors to messages of a batch as though they were each a batch of one message (similar to the for_each processor), but where each message is processed in...
- [parquet_decode](https://docs.redpanda.com/connect/components/processors/parquet_decode.md): Decodes Parquet files into a batch of structured messages.
- [parquet_encode](https://docs.redpanda.com/connect/components/processors/parquet_encode.md): Encodes Parquet files from a batch of structured messages.
- [parquet](https://docs.redpanda.com/connect/components/processors/parquet.md): Converts batches of documents to or from Parquet files.
- [parse_log](https://docs.redpanda.com/connect/components/processors/parse_log.md): Parses common log formats into structured data. This is easier and often much faster than grok.
- [processors](https://docs.redpanda.com/connect/components/processors/processors.md): A processor grouping several sub-processors.
- [protobuf](https://docs.redpanda.com/connect/components/processors/protobuf.md): Performs conversions to or from a protobuf message. This processor uses reflection, meaning conversions can be made directly from the target .proto files.
- [qdrant](https://docs.redpanda.com/connect/components/processors/qdrant.md): Query items within a Qdrant collection.
- [rate_limit](https://docs.redpanda.com/connect/components/processors/rate_limit.md): Throttles the throughput of a pipeline according to a specified rate_limit resource. Rate limits are shared across components and therefore apply globally to all processing pipelines.
- [redis_script](https://docs.redpanda.com/connect/components/processors/redis_script.md): Performs actions against Redis using LUA scripts.
- [redis](https://docs.redpanda.com/connect/components/processors/redis.md): Performs actions against Redis that aren't possible using a cache processor. Actions are performed for each message and the message contents are replaced with the result. In order to merge the result...
- [redpanda_data_transform](https://docs.redpanda.com/connect/components/processors/redpanda_data_transform.md): Executes a Redpanda Data Transform as a processor.
- [resource](https://docs.redpanda.com/connect/components/processors/resource.md): Resource is a processor type that runs a processor resource identified by its label.
- [retry](https://docs.redpanda.com/connect/components/processors/retry.md): Attempts to execute a series of child processors until success.
- [schema_registry_decode](https://docs.redpanda.com/connect/components/processors/schema_registry_decode.md): Automatically decodes and validates messages with schemas from a Confluent Schema Registry service.
- [schema_registry_encode](https://docs.redpanda.com/connect/components/processors/schema_registry_encode.md): Automatically encodes and validates messages with schemas from a Confluent Schema Registry service.
- [select_parts](https://docs.redpanda.com/connect/components/processors/select_parts.md): Cherry pick a set of messages from a batch by their index. Indexes larger than the number of messages are simply ignored.
- [sentry_capture](https://docs.redpanda.com/connect/components/processors/sentry_capture.md): Captures log events from messages and submits them to Sentry.
- [slack_thread](https://docs.redpanda.com/connect/components/processors/slack_thread.md): Reads a Slack thread using the Slack API method conversations.replies.
- [sleep](https://docs.redpanda.com/connect/components/processors/sleep.md): Sleep for a period of time specified as a duration string for each message. This processor will interpolate functions within the duration field, you can find a list of functions here.
- [split](https://docs.redpanda.com/connect/components/processors/split.md): Breaks message batches (synonymous with multiple part messages) into smaller batches. The size of the resulting batches are determined either by a discrete size or, if the field byte_size is...
- [sql_insert](https://docs.redpanda.com/connect/components/processors/sql_insert.md): Inserts rows into an SQL database for each message, and leaves the message unchanged.
- [sql_raw](https://docs.redpanda.com/connect/components/processors/sql_raw.md): Runs an arbitrary SQL query against a database and (optionally) returns the result as an array of objects, one for each row returned.
- [sql_select](https://docs.redpanda.com/connect/components/processors/sql_select.md): Runs an SQL select query against a database and returns the result as an array of objects, one for each row returned, containing a key for each column queried and its value.
- [sql](https://docs.redpanda.com/connect/components/processors/sql.md): Runs an arbitrary SQL query against a database and (optionally) returns the result as an array of objects, one for each row returned.
- [string_split](https://docs.redpanda.com/connect/components/processors/string_split.md): Splits a string by a delimiter into an array. Generally, using bloblang's split method is preferred. In some high performance use cases this processor can be faster than the equivalent bloblang if...
- [subprocess](https://docs.redpanda.com/connect/components/processors/subprocess.md): Executes a command as a subprocess and, for each message, will pipe its contents to the stdin stream of the process followed by a newline.
- [switch](https://docs.redpanda.com/connect/components/processors/switch.md): Conditionally processes messages based on their contents.
- [sync_response](https://docs.redpanda.com/connect/components/processors/sync_response.md): Adds the payload in its current state as a synchronous response to the input source, where it is dealt with according to that specific input type.
- [text_chunker](https://docs.redpanda.com/connect/components/processors/text_chunker.md): A processor that allows chunking and splitting text based on some strategy. Usually used for creating vector embeddings of large documents.
- [try_catch](https://docs.redpanda.com/connect/components/processors/try_catch.md): Executes a list of child processors on each message and, if any of them fail, executes a separate list of catch processors to recover from or react to the error.
- [try](https://docs.redpanda.com/connect/components/processors/try.md): Executes a list of child processors on messages only if no prior processors have failed (or the errors have been cleared).
- [unarchive](https://docs.redpanda.com/connect/components/processors/unarchive.md): Unarchives messages according to the selected archive format into multiple messages within a batch.
- [wasm](https://docs.redpanda.com/connect/components/processors/wasm.md): Executes a function exported by a WASM module for each message.
- [while](https://docs.redpanda.com/connect/components/processors/while.md): A processor that checks a Bloblang query against each batch of messages and executes child processors on them for as long as the query resolves to true.
- [workflow](https://docs.redpanda.com/connect/components/processors/workflow.md): Executes a topology of branch processors, performing them in parallel where possible.
- [xml](https://docs.redpanda.com/connect/components/processors/xml.md): Parses messages as an XML document, performs a mutation on the data, and then overwrites the previous contents with the new value.