hdfs
Type:
Reads files from a HDFS directory, where each discrete file will be consumed as a single message payload.
# Config fields, showing default values
input:
label: ""
hdfs:
hosts: [] # No default (required)
user: ""
directory: "" # No default (required)
Metadata
This input adds the following metadata fields to each message:
-
hdfs_name
-
hdfs_path
You can access these metadata fields using function interpolation.
Was this helpful?