Headers

• Connect-Protocol-Version number Required

  Define the version of the Connect protocol

  Value is 1.

• Connect-Timeout-Ms number

  Define the timeout, in ms

Responses

• 200 application/json

  Success

  Hide response attribute Show response attribute object

  • shadowLink object

    A ShadowLink resource

    Additional properties are NOT allowed.

    Hide shadowLink attributes Show shadowLink attributes object

    • configurations object

      ShadowLink options

      Additional properties are NOT allowed.

      Hide configurations attributes Show configurations attributes object

      • clientOptions object

        Options for the client link

        Additional properties are NOT allowed.

        Hide clientOptions attributes Show clientOptions attributes object

        • authenticationConfiguration object

          Additional properties are NOT allowed.

          Hide authenticationConfiguration attribute Show authenticationConfiguration attribute object

          • scramConfiguration object Required

            SCRAM settings

            Additional properties are NOT allowed.

            Hide scramConfiguration attributes Show scramConfiguration attributes object

            • passwordSet boolean

              Indicates that the password has been set

            • passwordSetAt string(date-time)

              A Timestamp represents a point in time independent of any time zone or local calendar, encoded as a count of seconds and fractions of seconds at nanosecond resolution. The count is relative to an epoch at UTC midnight on January 1, 1970, in the proleptic Gregorian calendar which extends the Gregorian calendar backwards to year one.

              All minutes are 60 seconds long. Leap seconds are "smeared" so that no leap second table is needed for interpretation, using a 24-hour linear smear.

              The range is from 0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting to that range, we ensure that we can convert to and from RFC 3339 date strings.

              # Examples

              Example 1: Compute Timestamp from POSIX time().

              Timestamp timestamp; timestamp.set_seconds(time(NULL)); timestamp.set_nanos(0);

              Example 2: Compute Timestamp from POSIX gettimeofday().

              struct timeval tv; gettimeofday(&tv, NULL);

              Timestamp timestamp; timestamp.set_seconds(tv.tv_sec); timestamp.set_nanos(tv.tv_usec * 1000);

              Example 3: Compute Timestamp from Win32 GetSystemTimeAsFileTime().

              FILETIME ft; GetSystemTimeAsFileTime(&ft); UINT64 ticks = (((UINT64)ft.dwHighDateTime) << 32) | ft.dwLowDateTime;

              // A Windows tick is 100 nanoseconds. Windows epoch 1601-01-01T00:00:00Z // is 11644473600 seconds before Unix epoch 1970-01-01T00:00:00Z. Timestamp timestamp; timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL)); timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));

              Example 4: Compute Timestamp from Java System.currentTimeMillis().

              long millis = System.currentTimeMillis();

              Timestamp timestamp = Timestamp.newBuilder().setSeconds(millis / 1000) .setNanos((int) ((millis % 1000) * 1000000)).build();

              Example 5: Compute Timestamp from Java Instant.now().

              Instant now = Instant.now();

              Timestamp timestamp = Timestamp.newBuilder().setSeconds(now.getEpochSecond()) .setNanos(now.getNano()).build();

              Example 6: Compute Timestamp from current time in Python.

              timestamp = Timestamp() timestamp.GetCurrentTime()

              # JSON Mapping

              In JSON format, the Timestamp type is encoded as a string in the RFC 3339 format. That is, the format is "{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z" where {year} is always expressed using four digits while {month}, {day}, {hour}, {min}, and {sec} are zero-padded to two digits each. The fractional seconds, which can go up to 9 digits (i.e. up to 1 nanosecond resolution), are optional. The "Z" suffix indicates the timezone ("UTC"); the timezone is required. A proto3 JSON serializer should always use UTC (as indicated by "Z") when printing the Timestamp type and a proto3 JSON parser should be able to accept both UTC and other timezones (as indicated by an offset).

              For example, "2017-01-15T01:30:15.01Z" encodes 15.01 seconds past 01:30 UTC on January 15, 2017.

              In JavaScript, one can convert a Date object to this format using the standard toISOString() method. In Python, a standard datetime.datetime object can be converted to this format using strftime with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in Java, one can use the Joda Time's ISODateTimeFormat.dateTime() to obtain a formatter capable of generating timestamps in this format.

            • scramMechanism string

              Valid SCRAM mechanisms

              Values are SCRAM_MECHANISM_UNSPECIFIED, SCRAM_MECHANISM_SCRAM_SHA_256, or SCRAM_MECHANISM_SCRAM_SHA_512.

            • username string

              SCRAM username

        • bootstrapServers array[string] Required

          The bootstrap servers to use

        • clientId string

          The Client ID for the Kafka RPC requests setn by this cluster to the source cluster

        • connectionTimeoutMs integer(int32)

          Connection timeout. If 0 is provided, defaults to 1 second

        • effectiveConnectionTimeoutMs integer(int32)

          The effective connection timeout in milliseconds

        • effectiveFetchMaxBytes integer(int32)

          The effective fetch max bytes

        • effectiveFetchMinBytes integer(int32)

          The effective fetch min bytes

        • effectiveFetchPartitionMaxBytes integer(int32)

          The effective fetch partition max bytes

        • effectiveFetchWaitMaxMs integer(int32)

          The effective fetch wait max in milliseconds

        • effectiveMetadataMaxAgeMs integer(int32)

          The effective metadata max age in milliseconds

        • effectiveRetryBackoffMs integer(int32)

          The effective retry backoff in milliseconds

        • fetchMaxBytes integer(int32)

          Fetch max bytes. If 0 is provided, defaults to 20 MiB

        • fetchMinBytes integer(int32)

          Fetch min bytes. If 0 is provided, defaults to 5 MiB

        • fetchPartitionMaxBytes integer(int32)

          Fetch partition max bytes. If 0 is provided, defaults to 1 MiB

        • fetchWaitMaxMs integer(int32)

          Fetch request timeout. If 0 is provided, defaults to 500ms

        • metadataMaxAgeMs integer(int32)

          Max metadata age. If 0 is provided, defaults to 10 seconds

        • retryBackoffMs integer(int32)

          Retry base backoff. If 0 is provided, defaults to 100ms

        • sourceClusterId string

          If provided, this is the expected ID of the source cluster. If it does not match then the connection will be rejected. If provided, this value must match the ClusterId field returned in the Kafka Metadata response message

        • tlsSettings object

          TLS settings

          One of:

          tls_file_settings object tls_pem_settings object

          TLS settings

          Hide attributes Show attributes

          • doNotSetSniHostname boolean

            If true, the SNI hostname will not be provided when TLS is used

          • enabled boolean

            Whether or not TLS is enabled

          • tlsFileSettings object Required

            TLS file settings

            Additional properties are NOT allowed.

            Hide tlsFileSettings attributes Show tlsFileSettings attributes object

            • caPath string

              Path to the CA

            • certPath string

              Path to the cert

            • keyPath string

              Key and Cert are optional but if one is provided, then both must be Path to the key

          TLS settings

          Hide attributes Show attributes

          • doNotSetSniHostname boolean

            If true, the SNI hostname will not be provided when TLS is used

          • enabled boolean

            Whether or not TLS is enabled

          • tlsPemSettings object Required

            Used when providing the TLS information in PEM format

            Additional properties are NOT allowed.

            Hide tlsPemSettings attributes Show tlsPemSettings attributes object

            • ca string

              The CA

            • cert string

              The cert

            • keyFingerprint string

              The SHA-256 of the key, in base64 format

      • consumerOffsetSyncOptions object

        Options for syncing consumer offsets

        Additional properties are NOT allowed.

        Hide consumerOffsetSyncOptions attributes Show consumerOffsetSyncOptions attributes object

        • effectiveInterval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • groupFilters array[object]

          The filters

          A filter based on the name of a resource

          Hide groupFilters attributes Show groupFilters attributes object

          • filterType string

            What type of filter this is, include or exclude

            Values are FILTER_TYPE_UNSPECIFIED, FILTER_TYPE_INCLUDE, or FILTER_TYPE_EXCLUDE.

          • name string

            The resource name, or "" Note if "", must be the only character and pattern_type must be PATTERN_TYPE_LITERAL

          • patternType string

            The matching pattern type

            Values are PATTERN_TYPE_UNSPECIFIED, PATTERN_TYPE_LITERAL, or PATTERN_TYPE_PREFIX.

        • interval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • paused boolean

          Allows user to pause the consumer offset sync task. If paused, then the task will enter the 'paused' state and not sync consumer offsets from the source cluster

      • schemaRegistrySyncOptions object

        Additional properties are NOT allowed.

        Hide schemaRegistrySyncOptions attribute Show schemaRegistrySyncOptions attribute object

        • shadowSchemaRegistryTopic object Required

          Shadow the entire source cluster's Schema Registry byte-for-byte. If set, the Shadow Link will attempt to add the _schemas topic to the list of Shadow Topics as long as:

          1. The _schemas topic exists on the source cluster
          2. The _schemas topic does not exist on the shadow cluster, or it is empty. If either of the above conditions are not met, then the _schemas topic will not be shadowed by this cluster. Unsetting this flag will not remove the _schemas topic from shadowing if it has already been added. Once made a shadow topic, the _schemas topic will be replicated byte-for-byte. To stop shadowing the _schemas topic, unset this field, then either fail-over the topic or delete it.

          Additional properties are NOT allowed.

      • securitySyncOptions object

        Options for syncing security settings

        Additional properties are NOT allowed.

        Hide securitySyncOptions attributes Show securitySyncOptions attributes object

        • aclFilters array[object]

          ACL filters

          A filter for ACLs

          Hide aclFilters attributes Show aclFilters attributes object

          • accessFilter object

            Filter an ACL based on its access

            Additional properties are NOT allowed.

            Hide accessFilter attributes Show accessFilter attributes object

            • host string

              The host to match. If not set, will default to match all hosts with the specified operation and permission_type. Note that the asterisk * is literal and matches hosts that are set to *

            • operation string

              / The ACL operation to match

              Values are ACL_OPERATION_UNSPECIFIED, ACL_OPERATION_ANY, ACL_OPERATION_READ, ACL_OPERATION_WRITE, ACL_OPERATION_CREATE, ACL_OPERATION_REMOVE, ACL_OPERATION_ALTER, ACL_OPERATION_DESCRIBE, ACL_OPERATION_CLUSTER_ACTION, ACL_OPERATION_DESCRIBE_CONFIGS, ACL_OPERATION_ALTER_CONFIGS, or ACL_OPERATION_IDEMPOTENT_WRITE.

            • permissionType string

              / ACL permission types

              Values are ACL_PERMISSION_TYPE_UNSPECIFIED, ACL_PERMISSION_TYPE_ANY, ACL_PERMISSION_TYPE_ALLOW, or ACL_PERMISSION_TYPE_DENY.

            • principal string

              The name of the principal, if not set will default to match all principals with the specified operation and permission_type

          • resourceFilter object

            A filter to match ACLs for resources

            Additional properties are NOT allowed.

            Hide resourceFilter attributes Show resourceFilter attributes object

            • name string

              Name, if not given will default to match all items in resource_type. Note that asterisk * is literal and matches resource ACLs that are named *

            • patternType string

              / The ACL pattern type

              Values are ACL_PATTERN_UNSPECIFIED, ACL_PATTERN_ANY, ACL_PATTERN_LITERAL, ACL_PATTERN_PREFIXED, or ACL_PATTERN_MATCH.

            • resourceType string

              / The ACL resource types

              Values are ACL_RESOURCE_UNSPECIFIED, ACL_RESOURCE_ANY, ACL_RESOURCE_CLUSTER, ACL_RESOURCE_GROUP, ACL_RESOURCE_TOPIC, ACL_RESOURCE_TXN_ID, ACL_RESOURCE_SR_SUBJECT, ACL_RESOURCE_SR_REGISTRY, or ACL_RESOURCE_SR_ANY.

        • effectiveInterval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • interval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • paused boolean

          Allows user to pause the security settings sync task. If paused, then the task will enter the 'paused' state and will not sync security settings from the source cluster

      • topicMetadataSyncOptions object

        Options for syncing topic metadata

        One of:

        start_at_earliest object start_at_latest object start_at_timestamp object

        Options for syncing topic metadata

        Hide attributes Show attributes

        • autoCreateShadowTopicFilters array[object]

          List of filters that indicate which topics should be automatically created as shadow topics on the shadow cluster. This only controls automatic creation of shadow topics and does not effect the state of the mirror topic once it is created. Literal filters for _consumer_offsets, _redpanda.audit_log and _schemas will be rejected as well as prefix filters to match topics prefixed with _redpanda or __redpanda. Wildcard * is permitted only for literal filters and will _not match any topics that start with _redpanda or __redpanda. If users wish to shadow topics that start with _redpanda or __redpanda, they should provide a literal filter for those topics.

          A filter based on the name of a resource

          Hide autoCreateShadowTopicFilters attributes Show autoCreateShadowTopicFilters attributes object

          • filterType string

            What type of filter this is, include or exclude

            Values are FILTER_TYPE_UNSPECIFIED, FILTER_TYPE_INCLUDE, or FILTER_TYPE_EXCLUDE.

          • name string

            The resource name, or "" Note if "", must be the only character and pattern_type must be PATTERN_TYPE_LITERAL

          • patternType string

            The matching pattern type

            Values are PATTERN_TYPE_UNSPECIFIED, PATTERN_TYPE_LITERAL, or PATTERN_TYPE_PREFIX.

        • effectiveInterval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • excludeDefault boolean

          If false, then the following topic properties will be synced by default:

          • compression.type
          • retention.bytes
          • retention.ms
          • delete.retention.ms
          • Replication Factor
          • min.compaction.lag.ms
          • max.compaction.lag.ms

          If this is true, then only the properties listed in synced_shadow_topic_properties will be synced.

        • interval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • paused boolean

          Allows user to pause the topic sync task. If paused, then the task will enter the 'paused' state and not sync topics or their properties from the source cluster

        • syncedShadowTopicProperties array[string]

          List of topic properties that should be synced from the source topic. The following properties will always be replicated

          • Partition count
          • max.message.bytes
          • cleanup.policy
          • timestamp.type

          The following properties are not allowed to be replicated and adding them to this list will result in an error:

          • redpanda.remote.readreplica
          • redpanda.remote.recovery
          • redpanda.remote.allowgaps
          • redpanda.virtual.cluster.id
          • redpanda.leaders.preference
          • redpanda.cloud_topic.enabled

          This list is a list of properties in addition to the default properties that will be synced. See exclude_default.

        • startAtEarliest object Required

          Start at the earliest offset in the partition.

          Additional properties are NOT allowed.

        Options for syncing topic metadata

        Hide attributes Show attributes

        • autoCreateShadowTopicFilters array[object]

          List of filters that indicate which topics should be automatically created as shadow topics on the shadow cluster. This only controls automatic creation of shadow topics and does not effect the state of the mirror topic once it is created. Literal filters for _consumer_offsets, _redpanda.audit_log and _schemas will be rejected as well as prefix filters to match topics prefixed with _redpanda or __redpanda. Wildcard * is permitted only for literal filters and will _not match any topics that start with _redpanda or __redpanda. If users wish to shadow topics that start with _redpanda or __redpanda, they should provide a literal filter for those topics.

          A filter based on the name of a resource

          Hide autoCreateShadowTopicFilters attributes Show autoCreateShadowTopicFilters attributes object

          • filterType string

            What type of filter this is, include or exclude

            Values are FILTER_TYPE_UNSPECIFIED, FILTER_TYPE_INCLUDE, or FILTER_TYPE_EXCLUDE.

          • name string

            The resource name, or "" Note if "", must be the only character and pattern_type must be PATTERN_TYPE_LITERAL

          • patternType string

            The matching pattern type

            Values are PATTERN_TYPE_UNSPECIFIED, PATTERN_TYPE_LITERAL, or PATTERN_TYPE_PREFIX.

        • effectiveInterval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • excludeDefault boolean

          If false, then the following topic properties will be synced by default:

          • compression.type
          • retention.bytes
          • retention.ms
          • delete.retention.ms
          • Replication Factor
          • min.compaction.lag.ms
          • max.compaction.lag.ms

          If this is true, then only the properties listed in synced_shadow_topic_properties will be synced.

        • interval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • paused boolean

          Allows user to pause the topic sync task. If paused, then the task will enter the 'paused' state and not sync topics or their properties from the source cluster

        • syncedShadowTopicProperties array[string]

          List of topic properties that should be synced from the source topic. The following properties will always be replicated

          • Partition count
          • max.message.bytes
          • cleanup.policy
          • timestamp.type

          The following properties are not allowed to be replicated and adding them to this list will result in an error:

          • redpanda.remote.readreplica
          • redpanda.remote.recovery
          • redpanda.remote.allowgaps
          • redpanda.virtual.cluster.id
          • redpanda.leaders.preference
          • redpanda.cloud_topic.enabled

          This list is a list of properties in addition to the default properties that will be synced. See exclude_default.

        • startAtLatest object Required

          Start at the latest offset in the partition.

          Additional properties are NOT allowed.

        Options for syncing topic metadata

        Hide attributes Show attributes

        • autoCreateShadowTopicFilters array[object]

          List of filters that indicate which topics should be automatically created as shadow topics on the shadow cluster. This only controls automatic creation of shadow topics and does not effect the state of the mirror topic once it is created. Literal filters for _consumer_offsets, _redpanda.audit_log and _schemas will be rejected as well as prefix filters to match topics prefixed with _redpanda or __redpanda. Wildcard * is permitted only for literal filters and will _not match any topics that start with _redpanda or __redpanda. If users wish to shadow topics that start with _redpanda or __redpanda, they should provide a literal filter for those topics.

          A filter based on the name of a resource

          Hide autoCreateShadowTopicFilters attributes Show autoCreateShadowTopicFilters attributes object

          • filterType string

            What type of filter this is, include or exclude

            Values are FILTER_TYPE_UNSPECIFIED, FILTER_TYPE_INCLUDE, or FILTER_TYPE_EXCLUDE.

          • name string

            The resource name, or "" Note if "", must be the only character and pattern_type must be PATTERN_TYPE_LITERAL

          • patternType string

            The matching pattern type

            Values are PATTERN_TYPE_UNSPECIFIED, PATTERN_TYPE_LITERAL, or PATTERN_TYPE_PREFIX.

        • effectiveInterval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • excludeDefault boolean

          If false, then the following topic properties will be synced by default:

          • compression.type
          • retention.bytes
          • retention.ms
          • delete.retention.ms
          • Replication Factor
          • min.compaction.lag.ms
          • max.compaction.lag.ms

          If this is true, then only the properties listed in synced_shadow_topic_properties will be synced.

        • interval string(duration)

          A Duration represents a signed, fixed-length span of time represented as a count of seconds and fractions of seconds at nanosecond resolution. It is independent of any calendar and concepts like "day" or "month". It is related to Timestamp in that the difference between two Timestamp values is a Duration and it can be added or subtracted from a Timestamp. Range is approximately +-10,000 years.

          # Examples

          Example 1: Compute Duration from two Timestamps in pseudo code.

          Timestamp start = ...; Timestamp end = ...; Duration duration = ...;

          duration.seconds = end.seconds - start.seconds; duration.nanos = end.nanos - start.nanos;

          if (duration.seconds < 0 && duration.nanos > 0) { duration.seconds += 1; duration.nanos -= 1000000000; } else if (duration.seconds > 0 && duration.nanos < 0) { duration.seconds -= 1; duration.nanos += 1000000000; }

          Example 2: Compute Timestamp from Timestamp + Duration in pseudo code.

          Timestamp start = ...; Duration duration = ...; Timestamp end = ...;

          end.seconds = start.seconds + duration.seconds; end.nanos = start.nanos + duration.nanos;

          if (end.nanos < 0) { end.seconds -= 1; end.nanos += 1000000000; } else if (end.nanos >= 1000000000) { end.seconds += 1; end.nanos -= 1000000000; }

          Example 3: Compute Duration from datetime.timedelta in Python.

          td = datetime.timedelta(days=3, minutes=10) duration = Duration() duration.FromTimedelta(td)

          # JSON Mapping

          In JSON format, the Duration type is encoded as a string rather than an object, where the string ends in the suffix "s" (indicating seconds) and is preceded by the number of seconds, with nanoseconds expressed as fractional seconds. For example, 3 seconds with 0 nanoseconds should be encoded in JSON format as "3s", while 3 seconds and 1 nanosecond should be expressed in JSON format as "3.000000001s", and 3 seconds and 1 microsecond should be expressed in JSON format as "3.000001s".

        • paused boolean

          Allows user to pause the topic sync task. If paused, then the task will enter the 'paused' state and not sync topics or their properties from the source cluster

        • syncedShadowTopicProperties array[string]

          List of topic properties that should be synced from the source topic. The following properties will always be replicated

          • Partition count
          • max.message.bytes
          • cleanup.policy
          • timestamp.type

          The following properties are not allowed to be replicated and adding them to this list will result in an error:

          • redpanda.remote.readreplica
          • redpanda.remote.recovery
          • redpanda.remote.allowgaps
          • redpanda.virtual.cluster.id
          • redpanda.leaders.preference
          • redpanda.cloud_topic.enabled

          This list is a list of properties in addition to the default properties that will be synced. See exclude_default.

        • startAtTimestamp string(date-time) Required

          A Timestamp represents a point in time independent of any time zone or local calendar, encoded as a count of seconds and fractions of seconds at nanosecond resolution. The count is relative to an epoch at UTC midnight on January 1, 1970, in the proleptic Gregorian calendar which extends the Gregorian calendar backwards to year one.

          All minutes are 60 seconds long. Leap seconds are "smeared" so that no leap second table is needed for interpretation, using a 24-hour linear smear.

          The range is from 0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting to that range, we ensure that we can convert to and from RFC 3339 date strings.

          # Examples

          Example 1: Compute Timestamp from POSIX time().

          Timestamp timestamp; timestamp.set_seconds(time(NULL)); timestamp.set_nanos(0);

          Example 2: Compute Timestamp from POSIX gettimeofday().

          struct timeval tv; gettimeofday(&tv, NULL);

          Timestamp timestamp; timestamp.set_seconds(tv.tv_sec); timestamp.set_nanos(tv.tv_usec * 1000);

          Example 3: Compute Timestamp from Win32 GetSystemTimeAsFileTime().

          FILETIME ft; GetSystemTimeAsFileTime(&ft); UINT64 ticks = (((UINT64)ft.dwHighDateTime) << 32) | ft.dwLowDateTime;

          // A Windows tick is 100 nanoseconds. Windows epoch 1601-01-01T00:00:00Z // is 11644473600 seconds before Unix epoch 1970-01-01T00:00:00Z. Timestamp timestamp; timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL)); timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));

          Example 4: Compute Timestamp from Java System.currentTimeMillis().

          long millis = System.currentTimeMillis();

          Timestamp timestamp = Timestamp.newBuilder().setSeconds(millis / 1000) .setNanos((int) ((millis % 1000) * 1000000)).build();

          Example 5: Compute Timestamp from Java Instant.now().

          Instant now = Instant.now();

          Timestamp timestamp = Timestamp.newBuilder().setSeconds(now.getEpochSecond()) .setNanos(now.getNano()).build();

          Example 6: Compute Timestamp from current time in Python.

          timestamp = Timestamp() timestamp.GetCurrentTime()

          # JSON Mapping

          In JSON format, the Timestamp type is encoded as a string in the RFC 3339 format. That is, the format is "{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z" where {year} is always expressed using four digits while {month}, {day}, {hour}, {min}, and {sec} are zero-padded to two digits each. The fractional seconds, which can go up to 9 digits (i.e. up to 1 nanosecond resolution), are optional. The "Z" suffix indicates the timezone ("UTC"); the timezone is required. A proto3 JSON serializer should always use UTC (as indicated by "Z") when printing the Timestamp type and a proto3 JSON parser should be able to accept both UTC and other timezones (as indicated by an offset).

          For example, "2017-01-15T01:30:15.01Z" encodes 15.01 seconds past 01:30 UTC on January 15, 2017.

          In JavaScript, one can convert a Date object to this format using the standard toISOString() method. In Python, a standard datetime.datetime object can be converted to this format using strftime with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in Java, one can use the Joda Time's ISODateTimeFormat.dateTime() to obtain a formatter capable of generating timestamps in this format.

    • name string Required

      The name of the shadow link

    • status object

      Status of the shadow link

      Additional properties are NOT allowed.

      Hide status attributes Show status attributes object

      • shadowTopics array[object]

        Status of shadow topics

        A ShadowTopic resource contained by the ShadowLink

        Hide shadowTopics attributes Show shadowTopics attributes object

        • name string

          Name of the shadow topic

        • sourceTopicId string

          The topic ID of the source topic

        • sourceTopicName string

          The name of the source topic

        • status object

          Status of a ShadowTopic

          Additional properties are NOT allowed.

          Hide status attributes Show status attributes object

          • partitionInformation array[object]

            List of partition information for the shadow topic

            Topic partition information

            Hide partitionInformation attributes Show partitionInformation attributes object

            • highWatermark integer | string

              Shadowed partition's HWM

            • partitionId integer | string

              Partition ID

            • sourceHighWatermark integer | string

              Source partition's HWM

            • sourceLastStableOffset integer | string

              Source partition's LSO

            • sourceLastUpdatedTimestamp string(date-time)

              A Timestamp represents a point in time independent of any time zone or local calendar, encoded as a count of seconds and fractions of seconds at nanosecond resolution. The count is relative to an epoch at UTC midnight on January 1, 1970, in the proleptic Gregorian calendar which extends the Gregorian calendar backwards to year one.

              All minutes are 60 seconds long. Leap seconds are "smeared" so that no leap second table is needed for interpretation, using a 24-hour linear smear.

              The range is from 0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting to that range, we ensure that we can convert to and from RFC 3339 date strings.

              # Examples

              Example 1: Compute Timestamp from POSIX time().

              Timestamp timestamp; timestamp.set_seconds(time(NULL)); timestamp.set_nanos(0);

              Example 2: Compute Timestamp from POSIX gettimeofday().

              struct timeval tv; gettimeofday(&tv, NULL);

              Timestamp timestamp; timestamp.set_seconds(tv.tv_sec); timestamp.set_nanos(tv.tv_usec * 1000);

              Example 3: Compute Timestamp from Win32 GetSystemTimeAsFileTime().

              FILETIME ft; GetSystemTimeAsFileTime(&ft); UINT64 ticks = (((UINT64)ft.dwHighDateTime) << 32) | ft.dwLowDateTime;

              // A Windows tick is 100 nanoseconds. Windows epoch 1601-01-01T00:00:00Z // is 11644473600 seconds before Unix epoch 1970-01-01T00:00:00Z. Timestamp timestamp; timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL)); timestamp.set_nanos((INT32) ((ticks % 10000000) * 100));

              Example 4: Compute Timestamp from Java System.currentTimeMillis().

              long millis = System.currentTimeMillis();

              Timestamp timestamp = Timestamp.newBuilder().setSeconds(millis / 1000) .setNanos((int) ((millis % 1000) * 1000000)).build();

              Example 5: Compute Timestamp from Java Instant.now().

              Instant now = Instant.now();

              Timestamp timestamp = Timestamp.newBuilder().setSeconds(now.getEpochSecond()) .setNanos(now.getNano()).build();

              Example 6: Compute Timestamp from current time in Python.

              timestamp = Timestamp() timestamp.GetCurrentTime()

              # JSON Mapping

              In JSON format, the Timestamp type is encoded as a string in the RFC 3339 format. That is, the format is "{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z" where {year} is always expressed using four digits while {month}, {day}, {hour}, {min}, and {sec} are zero-padded to two digits each. The fractional seconds, which can go up to 9 digits (i.e. up to 1 nanosecond resolution), are optional. The "Z" suffix indicates the timezone ("UTC"); the timezone is required. A proto3 JSON serializer should always use UTC (as indicated by "Z") when printing the Timestamp type and a proto3 JSON parser should be able to accept both UTC and other timezones (as indicated by an offset).

              For example, "2017-01-15T01:30:15.01Z" encodes 15.01 seconds past 01:30 UTC on January 15, 2017.

              In JavaScript, one can convert a Date object to this format using the standard toISOString() method. In Python, a standard datetime.datetime object can be converted to this format using strftime with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in Java, one can use the Joda Time's ISODateTimeFormat.dateTime() to obtain a formatter capable of generating timestamps in this format.

          • state string

            State of a shadow topic

            Values are SHADOW_TOPIC_STATE_UNSPECIFIED, SHADOW_TOPIC_STATE_ACTIVE, SHADOW_TOPIC_STATE_FAULTED, SHADOW_TOPIC_STATE_PAUSED, SHADOW_TOPIC_STATE_FAILING_OVER, SHADOW_TOPIC_STATE_FAILED_OVER, SHADOW_TOPIC_STATE_PROMOTING, or SHADOW_TOPIC_STATE_PROMOTED.

        • topicId string

          The topic ID of the shadow topic

      • state string

        State of the shadow link

        Values are SHADOW_LINK_STATE_UNSPECIFIED, SHADOW_LINK_STATE_ACTIVE, or SHADOW_LINK_STATE_PAUSED.

      • syncedShadowTopicProperties array[string]

        List of topic properties that are being synced

      • taskStatuses array[object]

        Statuses of the running tasks

        Status of a task

        Hide taskStatuses attributes Show taskStatuses attributes object

        • brokerId integer(int32)

          The broker the task is running on

        • name string

          Name of the task

        • reason string

          Reason for task being in state

        • shard integer(int32)

          The shard the task is running on

        • state string

          Task states

          Values are TASK_STATE_UNSPECIFIED, TASK_STATE_ACTIVE, TASK_STATE_PAUSED, TASK_STATE_LINK_UNAVAILABLE, TASK_STATE_NOT_RUNNING, or TASK_STATE_FAULTED.

    • uid string

      The UUID of the shadow link

• default application/json

  Error

  Hide response attributes Show response attributes object

  • code string

    The status code, which should be an enum value of [google.rpc.Code][google.rpc.Code].

    Values are canceled, unknown, invalid_argument, deadline_exceeded, not_found, already_exists, permission_denied, resource_exhausted, failed_precondition, aborted, out_of_range, unimplemented, internal, unavailable, data_loss, or unauthenticated.

  • details array[object]

    A list of messages that carry the error details. There is no limit on the number of messages.

    Contains an arbitrary serialized message along with a @type that describes the type of the serialized message, with an additional debug field for ConnectRPC error details.

    Hide details attributes Show details attributes object

    • debug object

      Detailed error information.

      Additional properties are allowed.

    • type string

      A URL that acts as a globally unique identifier for the type of the serialized message. For example: type.googleapis.com/google.rpc.ErrorInfo. This is used to determine the schema of the data in the value field and is the discriminator for the debug field.

    • value string(binary)

      The Protobuf message, serialized as bytes and base64-encoded. The specific message type is identified by the type field.

  • message string

    A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the [google.rpc.Status.details][google.rpc.Status.details] field, or localized by the client.