Apache Kafka Consumer group and Simple Consumer
apache-kafka
Solution
For the most part, the high-level consumer API does not let you control the offset directly.
When the consumer group is first created, you can tell it whether to start with the oldest or newest message that kafka has stored using the `auto.offset.reset` property.
You can also control when the high-level consumer commits new offsets to zookeeper by setting `auto.commit.enable` to false.
Since the high-level consumer stores the offsets in zookeeper, your app could access zookeeper directly and manipulate the offsets - but it would be outside of the high-level consumer API.
Your question was a little confusing but you can use the simple consumer in a multi-threaded environment. That's what the high-level consumer does.
Problem
I am new to Kafka, what I've understood sofar regarding the consumer is there are basically two types of implementation. 1) The High level consumer/consumer group 2) Simple Consumer The most important part about the high level abstraction is it used when Kafka doesn't care about handling the offset while the Simple consumer provides much better control over the offset management. What confuse me is what if I want to run consumer in a multithreaded environment and also want to have control over the offset.If I use consumer group does that mean I must read from the last offset stored in zookeeper? is that the only option I have.