This topic outlines how to add secrets to a Redpanda Connect config without exposing them.

Using environment variables

One of the most prolific approaches to providing secrets to a service is via environment variables. Redpanda Connect allows you to inject the values of environment variables into a configuration with the interpolation syntax ${FOO}, within a config it looks like this:

  super_secret: "${SECRET}"
Use quotes

Note that it would be valid to have super_secret: ${SECRET} above (without the quotes), but if SECRET is unset then the config becomes structurally different. Therefore, it’s always best to wrap environment variable interpolations with quotes so that when the variable is unset you still have a valid config (with an empty string).

More information about this syntax can be found on the interpolation field page.

Using CLI flags

As an alternative to environment variables it’s possible to set specific fields within a config using the CLI flag --set where the syntax is a <path>=<value> pair, the path being a dot-separated path to the field being set and the value being the thing to set it to. If, for example, we had the config:

  super_secret: ""

And we wanted to set the value of super_secret to a value stored within something like Hashicorp Vault we could run the config using the --set flag with backticks to execute a shell command for the value:

rpk connect run -c ./config.yaml \
  --set "thing.super_secret=`vault kv get -mount=secret thing_secret`"

Using this method we can inject the secret into the config without "leaking" it into an environment variable.

Avoiding leaked secrets

There are a few ways in which configs parsed by Redpanda Connect can be exported back out of the service. In all of these cases Redpanda Connect will attempt to scrub any field values within the config that are known secrets (any field marked as a secret in the docs).

However, if you’re embedding secrets within a config outside of the value of secret fields, maybe as part of a Bloblang mapping, then care should be made to avoid exposing the resulting config. This specifically means you should not enable debug HTTP endpoints when the port is exposed, and don’t use the rpk connect run echo subcommand on configs containing secrets unless you’re printing to a secure pipe.