Flume, Split data based on country in HDFS sink

Dear Folks,

In Flume HDFS sink, I have to split my incoming web logs data based upon country, how can we? where I configure this property.
Any sample sink config, please share the code.
Eg: If it is “India” data goes to India folder like that…


You have to write a custom interceptor in Java to get the output like that.
Sample: http://www.owenrumney.co.uk/2015/06/17/Flume-Interceptors.html

Do you want your program to read your file and move based on the data to the respective folder? or read file name and based on the name move the file to respective folders .

for both , It is not possible via flume default config . You have to write your own custom Interceptor to achieve this .

1 Like

Thanks Raj, will try it out