Docs Connect Components Inputs generate generate Available in: Cloud, Self-Managed Generates messages at a given interval using a Bloblang mapping executed without a context. This allows you to generate messages for testing your pipeline configs. Introduced in version 3.40.0. # Config fields, showing default values input: label: "" generate: mapping: root = "hello world" # No default (required) interval: 1s count: 0 batch_size: 1 auto_replay_nacks: true Examples Cron Scheduled Processing A common use case for the generate input is to trigger processors on a schedule so that the processors themselves can behave similarly to an input. The following configuration reads rows from a PostgreSQL table every 5 minutes. input: generate: interval: '@every 5m' mapping: 'root = {}' processors: - sql_select: driver: postgres dsn: postgres://foouser:foopass@localhost:5432/testdb?sslmode=disable table: foo columns: [ "*" ] Generate 100 Rows The generate input can be used as a convenient way to generate test data. The following example generates 100 rows of structured data by setting an explicit count. The interval field is set to empty, which means data is generated as fast as the downstream components can consume it. input: generate: count: 100 interval: "" mapping: | root = if random_int() % 2 == 0 { { "type": "foo", "foo": "is yummy" } } else { { "type": "bar", "bar": "is gross" } } Fields auto_replay_nacks Whether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent. If set to false these messages will instead be deleted. Disabling auto replays can greatly improve memory efficiency of high throughput streams as the original shape of the data can be discarded immediately upon consumption and mutation. Type: bool Default: true batch_size The number of generated messages that should be accumulated into each batch flushed at the specified interval. Type: int Default: 1 count An optional number of messages to generate, if set above 0 the specified number of messages is generated and then the input will shut down. Type: int Default: 0 interval The time interval at which messages should be generated, expressed either as a duration string or as a cron expression. If set to an empty string messages will be generated as fast as downstream services can process them. Cron expressions can specify a timezone by prefixing the expression with TZ=<location name>, where the location name corresponds to a file within the IANA Time Zone database. Type: string Default: 1s # Examples: interval: 5s interval: 1m interval: 1h interval: @every 1s interval: 0,30 */2 * * * * interval: TZ=Europe/London 30 3-6,20-23 * * * mapping A Bloblang mapping to use for generating messages. Type: string # Examples: mapping: root = "hello world" mapping: root = {"test":"message","id":uuid_v4()} Back to top × Simple online edits For simple changes, such as fixing a typo, you can edit the content directly on GitHub. Edit on GitHub Or, open an issue to let us know about something that you want us to change. Open an issue Contribution guide For extensive content updates, or if you prefer to work locally, read our contribution guide . Was this helpful? thumb_up thumb_down group Ask in the community mail Share your feedback group_add Make a contribution 🎉 Thanks for your feedback! gcp_spanner_cdc git