Getting issue while running flume conf file

Have run following command
flume-ng agent --name a1 --conf /home/jayantm1988/flumeExample -f multiplexingExampleNew.conf

Issue what am getting
17/04/23 03:26:49 ERROR node.Application: A fatal error occurred while running. Exception follows.
org.apache.commons.cli.ParseException: The specified configuration file does not exist: /home/jayantm1988/multiplexingExampleNew.conf

Don’t understand why is it searching conf file in ‘/home/jayantm1988’ , have give conf folder location /home/jayantm1988/flumeExample

Please help me ASAP

Here is my conf file code

Name the components on this agent

a1.sources = r1
a1.sinks = k1 k2
a1.channels = c1 c2

Describe the source

a1.sources.r1.type = exec
a1.sources.r1.command = tail -f /opt/gen_logs/logs/access.log

Describe sink

a1.sinks.k1.type = logger

Describe sink

a1.sinks.k2.type = hdfs
a1.sinks.k2.hdfs.path = hdfs://nn01.itversity.com:8020/user/dgadiraju/wlabssa/flume

Describe channel c1

a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

Describe channel c1

a1.channels.c2.type = file
a1.channels.c2.capacity = 1000
a1.channels.c2.transactionCapacity = 100

Bind source and sink to channel

a1.sources.r1.channels = c1 c2
a1.sinks.k1.channels = c1
a1.sinks.k2.channels = c2

Try giving full path for file name -

flume-ng agent --name a1 --conf /home/jayantm1988/flumeExample -f /home/jayantm1988/flumeExample/multiplexingExampleNew.conf

Also try below command -

flume-ng agent --name a1 -c /home/jayantm1988/flumeExample -f multiplexingExampleNew.conf

2 Likes

Yeah , now it’s working

Thanks

Hi,

I am getting following error while running flume-ng.

Here is the command I am using: flume-ng agent --name a1 --conf /home/rajeshv28/wlabssa --conf-file example.conf

17/05/15 15:38:39 INFO node.Application: Starting Channel c1
17/05/15 15:38:39 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: CHANNEL, name: c1: Successfully registered new MBean.
17/05/15 15:38:39 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: c1 started
17/05/15 15:38:39 INFO node.Application: Starting Sink k1
17/05/15 15:38:39 INFO node.Application: Starting Source r1
17/05/15 15:38:39 INFO source.NetcatSource: Source starting
17/05/15 15:38:40 ERROR source.NetcatSource: Unable to bind to socket. Exception follows.
java.net.SocketException: Unresolved address
at sun.nio.ch.Net.translateToSocketException(Net.java:131)
at sun.nio.ch.Net.translateException(Net.java:157)
at sun.nio.ch.Net.translateException(Net.java:163)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:76)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at org.apache.flume.source.NetcatSource.start(NetcatSource.java:162)
at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:101)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:218)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
… 11 more
17/05/15 15:38:40 ERROR lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:org.apache.flume.source.NetcatSource{name:r1,state:IDLE} } - Exception follows.
org.apache.flume.FlumeException: java.net.SocketException: Unresolved address
at org.apache.flume.source.NetcatSource.start(NetcatSource.java:168)
at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

Here is my agent:

example.conf: A single-node Flume configuration

Name the components on this agent

a1.sources = r1
a1.sinks = k1
a1.channels = c1

Describe/configure the source

a1.sources.r1.type = netcat
a1.sources.r1.bind = gw01.itversity.comqw
a1.sources.r1.port = 44444

Describe the sink

a1.sinks.k1.type = logger

Use a channel which buffers events in memory

a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

Bind the source and sink to the channel

a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

Ignore my post. its working. I gave wrong source bind.

Thank you.