Not able to run flume agent

flume
#1

Hi,
I am unable to run flume agent. This is my first program in flume. I am using the config file /etc/flume/conf/flume-conf.properties.template with no change in the default settings. The following command was issued

flume-ng agent -n agent --conf-file /etc/flume/conf/flume-conf.properties.template

But the system is throwing the following error and does not show anything thereafter. Neither it comes back to the command prompt until ^C pressed

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property ‘log4j2.debug’ to show Log4j2 internal initialization logging.

Please help

Thanks

0 Likes

#2

include conf file as part of the flume agent command

0 Likes

#3

Hi Team,

when i run

flume-ng agent -n wh -f /home/kishoresoft/flume_demo/wslogstohdfs/wshdfs.conf --conf /etc/flume/conf

i am getting error

ERROR RollingFileManager (/tmp/flume/flume.log) java.io.FileNotFoundException: /tmp/flume/flume.log (Permission denied) java.io.FileNotFoundExce
ption: /tmp/flume/flume.log (Permission denied)

2019-04-01 21:24:15,399 main ERROR Could not create plugin of type class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.Illega
lStateException: ManagerFactory [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory@5552768b] unable to create manager for [/tmp/flu
me/flume.log] with data [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData@3c947bc5[pattern=/tmp/flume/archive/flume.log.%d{yyyyMMdd}-%i, append=
true, bufferedIO=true, bufferSize=8192, policy=CompositeTriggeringPolicy(policies=[SizeBasedTriggeringPolicy(size=104857600), CronTriggeringPolicy(schedule=0 0 0 * * ?)]), s
trategy=DefaultRolloverStrategy(min=1, max=20, useMax=true), advertiseURI=null, layout=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %equals{%x}{[]}{} - %m%n, filePermis
sions=null, fileOwner=null]] java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory@555
2768b] unable to create manager for [/tmp/flume/flume.log] with data [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData@3c947bc5[pattern=/tmp/flu
me/archive/flume.log.%d{yyyyMMdd}-%i, append=true, bufferedIO=true, bufferSize=8192, policy=CompositeTriggeringPolicy(policies=[SizeBasedTriggeringPolicy(size=104857600), Cr
onTriggeringPolicy(schedule=0 0 0 * * ?)]), strategy=DefaultRolloverStrategy(min=1, max=20, useMax=true), advertiseURI=null, layout=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%
C.%M:%L) %equals{%x}{[]}{} - %m%n, filePermissions=null, fileOwner=null]]

2019-04-01 21:24:15,401 main ERROR Unable to invoke factory method in class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.Ill
egalStateException: No factory method found for class org.apache.logging.log4j.core.appender.RollingFileAppender java.lang.IllegalStateException: No factory method found for
class org.apache.logging.log4j.core.appender.RollingFileAppender

2019-04-01 21:24:15,403 main ERROR Null object returned for RollingFile in Appenders.

Can anyone help me in this.

0 Likes

#4

There are few typos in your conf file I have changed them and started the flume agent.
You have used hdfs path as http ://nn01.itversity.com:8020 instead of hdfs://nn01.itversity.com:8020

0 Likes

#5

Sorry for the mistake. Thank you Sunil. But still i am getting Errors.

ERROR RollingFileManager (/tmp/flume/flume.log) java.io.FileNotFoundException: /tmp/flume/flume.log (Permission denied) java.io.FileNotFoundExce
ption: /tmp/flume/flume.log (Permission denied)

2019-04-02 12:12:11,479 main ERROR Could not create plugin of type class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.Illega

lStateException: ManagerFactory [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory@5552768b] unable to create manager for [/tmp/flu
me/flume.log] with data [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData@3c947bc5[pattern=/tmp/flume/archive/flume.log.%d{yyyyMMdd}-%i, append=
true, bufferedIO=true, bufferSize=8192, policy=CompositeTriggeringPolicy(policies=[SizeBasedTriggeringPolicy(size=104857600), CronTriggeringPolicy(schedule=0 0 0 * * ?)]), s
trategy=DefaultRolloverStrategy(min=1, max=20, useMax=true), advertiseURI=null, layout=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %equals{%x}{[]}{} - %m%n, filePermis
sions=null, fileOwner=null]] java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory@555
2768b] unable to create manager for [/tmp/flume/flume.log] with data [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData@3c947bc5[pattern=/tmp/flu
me/archive/flume.log.%d{yyyyMMdd}-%i, append=true, bufferedIO=true, bufferSize=8192, policy=CompositeTriggeringPolicy(policies=[SizeBasedTriggeringPolicy(size=104857600), Cr
onTriggeringPolicy(schedule=0 0 0 * * ?)]), strategy=DefaultRolloverStrategy(min=1, max=20, useMax=true), advertiseURI=null, layout=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%
C.%M:%L) %equals{%x}{[]}{} - %m%n, filePermissions=null, fileOwner=null]]

2019-04-02 12:12:11,481 main ERROR Unable to invoke factory method in class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.Ill
egalStateException: No factory method found for class org.apache.logging.log4j.core.appender.RollingFileAppender java.lang.IllegalStateException: No factory method found for
class org.apache.logging.log4j.core.appender.RollingFileAppender

2019-04-02 12:12:11,482 main ERROR Null object returned for RollingFile in Appenders.

in my wshdfs.conf i given path

hdfs://nn01.itversity.com:8020/user/kishoresoft/flume_demo

but when i open file it is showing

hdfs://nn01.itversity.com:8020/user/kishoresoft/flume_deemo

the folder name is flume_deemo instead of flume_demo. i removed “e” and i did save and exit (:x) . But when i open the wshdfs.conf file still i am seeing “flume_deemo” and i am getting the above Error. Can you please fix this.

Thanks!

0 Likes

#6

You can ignore those errors and check whether the files are created or not at the HDFS location.

0 Likes