Flume - Web Server logs to HDFS


Originally published at: http://www.itversity.com/topic/cca175-flume-web-server-logs-to-hdfs-scala/

Let us see how we can set up Flume agent to get data from web server logs into HDFS Define Source – exec type Let us read data from web server logs. Use exec type as source Issue tail -F command to continuously read messages from the simulated logs Define Sink – HDFS type Let…