Difference between Kafka and Flume?

#1

Hi all, @itversity

What is the difference between Apache Kafka and Flume?

0 Likes

#2

1.
Kafka : It is a publish-subscribe messaging system, which offers strong durability, scalabitity and fault-tolerance support.

Flume : Flume is a distributed, reliable, and available system for efficiently collecting, aggregating, and moving large amounts of data from many different sources to a centralized data store, such as HDFS

2.
Kafka : Use Kafka if you need a highly reliable and scalable enterprise messaging system to connect many multiple systems, one of which is Hadoop.

Flume : Use Flume if you have an non-relational data sources such as log files that you want to stream into Hadoop.

3.
Kafka : High availability of events(recoverable in case of failures)

Flume : Flume does not replicate events - in case of flume-agent failure, you will lose events in the channel Flume does not replicate events - in case of flume-agent failure, you will lose events in the channel

2 Likes