Flume Error: Avro source - Required parameter bind must exist

Flume Error: Avro source - Required parameter bind must exist
0.0 0


Hi All,

I am trying to ingest logs from remote server(Azure VM) on to my Cloudera quickstart VM and have done the following :

Agent 1(to be run on remote server) - Exec source, file channel and Avro sink
Agent 2 (to be run on quickstart vm) - Avro source, file channel, hdfs sink

I am not able to check Agent1 since Agent 2 itself is throwing error.
Error is “Source avro-source has been removed due to an error during configuration” while I initiate Agent 2. Additionally it says Java exception “Required parameter bind must exist and may not be null”

Can anyone help in solving this. I am not able to figure out the issue in conf file.

Conf file on Quickstart vm(for Agent 2):

Name the components on this agent

Agent2.sources = avro-source
Agent2.channels = file-channel
Agent2.sinks = hdfs-sink

Describe/configure Source

Agent2.sources.avro-source.type = avro
Agent2.sources.avro-source.hostname = public IP of my quickstart VM
Agent2.sources.avro-source.port = 11111

Describe the sink

Agent2.sinks.hdfs-sink.type = hdfs
Agent2.sinks.hdfs-sink.hdfs.path = hdfs://IP of my VM:8020/user/cloudera/flumetest/Controller_%Y-%m-%d
Agent2.sinks.hdfs-sink.hdfs.rollInterval = 120
Agent2.sinks.hdfs-sink.hdfs.rollSize = 10485760
Agent2.sinks.hdfs-sink.hdfs.rollCount = 10000
Agent2.sinks.hdfs-sink.hdfs.fileType = DataStream
Agent2.sinks.hdfs-sink.hdfs.filePrefix = log
Agent2.sinks.hdfs-sink.hdfs.fileSuffix = .txt
Agent2.sinks.hdfs-sink.hdfs.inUseSuffix = .tmp
Agent2.sinks.hdfs-sink.hdfs.useLocalTimeStamp = true
#Use a channel which buffers events in file
Agent2.channels.file-channel.type = file
Agent2.channels.file-channel.checkpointDir = /home/cloudera/Projects/flumetest/checkpoint/
Agent2.channels.file-channel.dataDirs = /home/cloudera/Projects/flumetest/data/

Bind the source and sink to the channel

Agent2.sources.avro-source.channels = file-channel
Agent2.sinks.hdfs-sink.channel = file-channel