Kafka Consumer startup error: Failed to add leader for partitions [calls,0] - NotLeaderForPartitionException
apache-kafka, apache-zookeeper
Solution
A search shows that now, 7 years later, this is no longer a known problem for current versions.
There have also been multiple patches which resolve errors that may or may not be the same, and it is almost certain that one of these fixed the issue.
As such the only practical solution for anyone seems to be upgrading to a newer version. (For Kafka, Zookeeper as well as Windows.)
If the problem persists in currently relevant versions please ask a new question as the root cause is unlikely to be the same.
Problem
Note: this is for an older version : We are running kafka_2.9.2-0.8.1 When attempting to run the kafka-console-consumer.bat against an existing topic on windows7 we receive "failed to add leader" and "NotLeaderForPartition" exceptions Here is the command line ``` set GROUP=group1234 kafka-console-consumer.bat --group %GROUP% --zookeeper localhost:2181 --topic calls --from-beginning ``` Here are the errors: ``` [2014-05-26 15:02:12,997] WARN [group1234_S80035683-SC01-1401141732400-98745e28-leader-finder-thread], Failed to add leader for partitions [calls,0]; will retry (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread:89) kafka.common.NotLeaderForPartitionException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at kafka.common.ErrorMapping$.exceptionFor(ErrorMapping.scala:73) at kafka.consumer.SimpleConsumer.earliestOrLatestOffset(SimpleConsumer.scala:160) at kafka.consumer.ConsumerFetcherThread.handleOffsetOutOfRange(ConsumerFetcherThread.scala:60) at kafka.server.AbstractFetcherThread$$anonfun$addPartitions$2.apply(AbstractFetcherThread.scala:179) at kafka.server.AbstractFetcherThread$$anonfun$addPartitions$2.apply(AbstractFetcherThread.scala:174) at scala.collection.immutable.Map$Map1.foreach(Map.scala:119) at kafka.server.AbstractFetcherThread.addPartitions(AbstractFetcherThread.scala:174) at kafka.server.AbstractFetcherManager$$anonfun$addFetcherForPartitions$2.apply(AbstractFetcherManager.scala:86) at kafka.server.AbstractFetcherManager$$anonfun$addFetcherForPartitions$2.apply(AbstractFetcherManager.scala:76) at scala.collection.immutable.Map$Map1.foreach(Map.scala:119) at kafka.server.AbstractFetcherManager.addFetcherForPartitions(AbstractFetcherManager.scala:76) ``` And we are unable to consume any messages in the end. We are running kafka_2.9.2-0.8.1 Note: Zookeeper is seeing the Consumer attempt to connect: I go into zkCli and can see the new group1234 ``` [zk: localhost:2181(CONNECTED) 2] ls2 /consumers/group1234 [owners, ids] cZxid = 0x123 ctime = Mon May 26 15:02:12 PDT 2014 mZxid = 0x123 mtime = Mon May 26 15:02:12 PDT 2014 pZxid = 0x128 cversion = 2 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 2 ``` And here is info on the requested calls topic in ZK: ``` [zk: localhost:2181(CONNECTED) 7] ls2 /brokers/topics/calls [partitions] cZxid = 0x18 ctime = Sat May 24 23:15:16 PDT 2014 mZxid = 0x18 mtime = Sat May 24 23:15:16 PDT 2014 pZxid = 0x1c cversion = 1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 36 numChildren = 1 ``` In case there were corruption on the topic, I just dropped it in ZK and then recreated it via kafka-topics.bat. Here is the new ZK output ``` [zk: localhost:2181(CONNECTED) 15] ls2 /brokers/topics/calls [] cZxid = 0x136 ctime = Mon May 26 16:02:51 PDT 2014 mZxid = 0x136 mtime = Mon May 26 16:02:51 PDT 2014 pZxid = 0x136 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 36 numChildren = 0 ```