Kafka Broker-Connection to node -2 could not be established. Broker may not be available

Hi Team, Please assist on below issue.

I have followed this url to work on Kafka–> https://gist.github.com/balusai1/f7410a2835e0ddfd2721da20e90f1c42

I am able to create kafka-topic and produce the message but not able to consume from the created Topic(When i run the consumer, the cursor is just blinking ).

Then i passed below servers for both Producer and Consumer as they are in server.properties of Kafka/config
nn01.itversity.com:6667,nn02.itversity.com:6667,rm01.itversity.com:6667

Now i am getting below error from Both Producer and Consumer end.

[2020-07-28 13:14:46,276] WARN [Producer clientId=console-producer] Connection to node -2 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-07-28 13:14:46,278] WARN [Producer clientId=console-producer] Connection to node -3 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

[2020-07-28 13:14:36,333] WARN [Consumer clientId=consumer-1, groupId=console-consumer-27489] Connection to node -3 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2020-07-28 13:14:36,337] WARN [Consumer clientId=consumer-1, groupId=console-consumer-27489] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

@Chandu_P
Can you please share the command you tried?

Hi Hemanth,

I have executed below commands in TWO ways by changing SERVER HOST NAMES. Please check and provide the detailed commands.

Run every time you login or update .bash_profile

export KAFKA_HOME=/usr/hdp/current/kafka-broker
export PATH=$PATH:$KAFKA_HOME/bin

  1. When i tried below commands ,the CURSOR was just blinking. Below step are given for one of the Kafka quey on Itversity Lab (https://gist.github.com/balusai1/f7410a2835e0ddfd2721da20e90f1c42)

Create topics

kafka-topics.sh --create
–zookeeper nn01.itversity.com:2181,nn02.itversity.com:2181,rm01.itversity.com:2181
–replication-factor 1
–partitions 1
–topic mynewtopic2

Command to produce messages, start typing after running this kakfa-console-producer command

The messages will be stored in topic kafkadg on the host where brokers are running

kafka-console-producer.sh
–broker-list wn01.itversity.com:6667,wn02.itversity.com:6667,wn03.itversity.com:6667
–topic mynewtopic2

Open another shell and then run kafka-console-consumer command to see streaming messages

kafka-console-consumer.sh
–bootstrap-server wn01.itversity.com:6667,wn02.itversity.com:6667,wn03.itversity.com:6667
–topic mynewtopic2
–from-beginning

===============================================================================================
2. When i tried below commands with the SERVER NAMES configured in kafka server.properties getting the error saying the brokers may not be available.

Create topics

kafka-topics.sh --create
–zookeeper nn01.itversity.com:2181,nn02.itversity.com:2181,rm01.itversity.com:2181
–replication-factor 1
–partitions 1
–topic venkytopic2

Command to produce messages, start typing after running this kakfa-console-producer command

The messages will be stored in topic kafkadg on the host where brokers are running

kafka-console-producer.sh
–broker-list nn01.itversity.com:6667,nn02.itversity.com:6667,rm03.itversity.com:6667
–topic venkytopic2

Open another shell and then run kafka-console-consumer command to see streaming messages

kafka-console-consumer.sh
–bootstrap-server nn01.itversity.com:6667,nn02.itversity.com:6667,rm03.itversity.com:6667
–topic venkytopic2
–from-beginning

Hi Team,

Please check the above method i followed .Please assist ASAP.

@Chandu_P
Thanks for your patience. The issue is resolved, can you please try now