kafka consumer not reporting to consumer group
we have setup a single node kafka cluster and running the consumers app on other server(around 10 consumers running). When i am trying to get the consumer details from kafka command line i can see few consumers are not reporting to the consumer as i get just the headers of the command output and not the actual kafka consumer details.
user@kafkaserver:/home/kafka/kafka$ bin/kafka-consumer-groups.sh --bootstrap-server <brokerip>:9092 --describe --group <group_name>
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
Server on which consumers are running is able to connect to broker server on port 9092 and also to zookeeper. Could this be an issue on the consumer?
apache-kafka kafka-consumer-api
asked Nov 12 '18 at 13:17
Parth Gandhi
347
add a comment |
we have setup a single node kafka cluster and running the consumers app on other server(around 10 consumers running). When i am trying to get the consumer details from kafka command line i can see few consumers are not reporting to the consumer as i get just the headers of the command output and not the actual kafka consumer details.
user@kafkaserver:/home/kafka/kafka$ bin/kafka-consumer-groups.sh --bootstrap-server <brokerip>:9092 --describe --group <group_name>
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
Server on which consumers are running is able to connect to broker server on port 9092 and also to zookeeper. Could this be an issue on the consumer?
apache-kafka kafka-consumer-api
asked Nov 12 '18 at 13:17
Parth Gandhi
347
Are you passing a validgroup-name? You must provide some details regarding your topic (e.g. how many partitions) and your consumers in order to be able to help you out.
– Giorgos Myrianthous
Nov 12 '18 at 13:19
What do you mean "also to Zookeeper"? As of Kafka 0.9, consumers really shouldn't be using Zookeeper. In other words, what consumer code do you have?
– cricket_007
Nov 12 '18 at 13:41
Its a UAT environment- all topics with single partition and a single consumer running.
– Parth Gandhi
Nov 12 '18 at 13:43
Is it single consumer or multiple consumers (as mentioned around 10 above)? are you using the group.id config you have used for this app in place of <group_name>?
– AbhishekN
Nov 12 '18 at 23:48
add a comment |
we have setup a single node kafka cluster and running the consumers app on other server(around 10 consumers running). When i am trying to get the consumer details from kafka command line i can see few consumers are not reporting to the consumer as i get just the headers of the command output and not the actual kafka consumer details.
user@kafkaserver:/home/kafka/kafka$ bin/kafka-consumer-groups.sh --bootstrap-server <brokerip>:9092 --describe --group <group_name>
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
Server on which consumers are running is able to connect to broker server on port 9092 and also to zookeeper. Could this be an issue on the consumer?
apache-kafka kafka-consumer-api
asked Nov 12 '18 at 13:17
Parth Gandhi
347
we have setup a single node kafka cluster and running the consumers app on other server(around 10 consumers running). When i am trying to get the consumer details from kafka command line i can see few consumers are not reporting to the consumer as i get just the headers of the command output and not the actual kafka consumer details.
user@kafkaserver:/home/kafka/kafka$ bin/kafka-consumer-groups.sh --bootstrap-server <brokerip>:9092 --describe --group <group_name>
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
Server on which consumers are running is able to connect to broker server on port 9092 and also to zookeeper. Could this be an issue on the consumer?
apache-kafka kafka-consumer-api
apache-kafka kafka-consumer-api
asked Nov 12 '18 at 13:17
Parth Gandhi
347
asked Nov 12 '18 at 13:17
Parth Gandhi
347
asked Nov 12 '18 at 13:17
Parth Gandhi
347
asked Nov 12 '18 at 13:17
Parth Gandhi
347
asked Nov 12 '18 at 13:17
Parth Gandhi
347
347
Are you passing a validgroup-name? You must provide some details regarding your topic (e.g. how many partitions) and your consumers in order to be able to help you out.
– Giorgos Myrianthous
Nov 12 '18 at 13:19
What do you mean "also to Zookeeper"? As of Kafka 0.9, consumers really shouldn't be using Zookeeper. In other words, what consumer code do you have?
– cricket_007
Nov 12 '18 at 13:41
Its a UAT environment- all topics with single partition and a single consumer running.
– Parth Gandhi
Nov 12 '18 at 13:43
Is it single consumer or multiple consumers (as mentioned around 10 above)? are you using the group.id config you have used for this app in place of <group_name>?
– AbhishekN
Nov 12 '18 at 23:48
add a comment |
Are you passing a validgroup-name? You must provide some details regarding your topic (e.g. how many partitions) and your consumers in order to be able to help you out.
– Giorgos Myrianthous
Nov 12 '18 at 13:19
What do you mean "also to Zookeeper"? As of Kafka 0.9, consumers really shouldn't be using Zookeeper. In other words, what consumer code do you have?
– cricket_007
Nov 12 '18 at 13:41
Its a UAT environment- all topics with single partition and a single consumer running.
– Parth Gandhi
Nov 12 '18 at 13:43
Is it single consumer or multiple consumers (as mentioned around 10 above)? are you using the group.id config you have used for this app in place of <group_name>?
– AbhishekN
Nov 12 '18 at 23:48
Are you passing a valid
group-name? You must provide some details regarding your topic (e.g. how many partitions) and your consumers in order to be able to help you out.– Giorgos Myrianthous
Nov 12 '18 at 13:19
Are you passing a valid
group-name? You must provide some details regarding your topic (e.g. how many partitions) and your consumers in order to be able to help you out.– Giorgos Myrianthous
Nov 12 '18 at 13:19
What do you mean "also to Zookeeper"? As of Kafka 0.9, consumers really shouldn't be using Zookeeper. In other words, what consumer code do you have?
– cricket_007
Nov 12 '18 at 13:41
What do you mean "also to Zookeeper"? As of Kafka 0.9, consumers really shouldn't be using Zookeeper. In other words, what consumer code do you have?
– cricket_007
Nov 12 '18 at 13:41
Its a UAT environment- all topics with single partition and a single consumer running.
– Parth Gandhi
Nov 12 '18 at 13:43
Its a UAT environment- all topics with single partition and a single consumer running.
– Parth Gandhi
Nov 12 '18 at 13:43
Is it single consumer or multiple consumers (as mentioned around 10 above)? are you using the group.id config you have used for this app in place of <group_name>?
– AbhishekN
Nov 12 '18 at 23:48
Is it single consumer or multiple consumers (as mentioned around 10 above)? are you using the group.id config you have used for this app in place of <group_name>?
– AbhishekN
Nov 12 '18 at 23:48
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53263008%2fkafka-consumer-not-reporting-to-consumer-group%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53263008%2fkafka-consumer-not-reporting-to-consumer-group%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Are you passing a valid
group-name? You must provide some details regarding your topic (e.g. how many partitions) and your consumers in order to be able to help you out.– Giorgos Myrianthous
Nov 12 '18 at 13:19
What do you mean "also to Zookeeper"? As of Kafka 0.9, consumers really shouldn't be using Zookeeper. In other words, what consumer code do you have?
– cricket_007
Nov 12 '18 at 13:41
Its a UAT environment- all topics with single partition and a single consumer running.
– Parth Gandhi
Nov 12 '18 at 13:43
Is it single consumer or multiple consumers (as mentioned around 10 above)? are you using the group.id config you have used for this app in place of <group_name>?
– AbhishekN
Nov 12 '18 at 23:48