Show Menu

kafka Cheat Sheet (DRAFT) by

This is a draft cheat sheet. It is a work in progress and is not finished yet.

Describe configs for a topic

bin/ka­fka­-co­nfi­ --zook­eeper localh­ost­:2181 --describe --enti­ty-type topics --enti­ty-name test_topic

Set retention times

bin/ka­fka­-to­ --zook­eeper localh­ost­:2181 --alter --topic test_topic --config retent­­=1000 # Deprecated way
bin/ka­fka­-co­nfi­ --zook­eeper localh­ost­:2181 --alter --enti­ty-type topics --enti­ty-name test_topic --add-­config retent­­=1000 # Modern way
If you need to delete all messages in topic, you can exploit the retention times. First set the retention time to something very low (1000 ms), wait a few seconds, then revert the retention times back to the previous value.

Note: The default retention time is 24 hours (86400000 millis).

Delete a topic

bin/ka­fka­-to­ --zook­eeper localh­ost­:2181 --delete --topic test_topic

Describe a topic

bin/ka­fka­-to­ --describe --zook­eeper localh­ost­:2181 --topic test_topic

Add a partition

bin/ka­fka­-to­ --alter --zook­eeper localh­ost­:2181 --topic test_topic --part­itions 3

Create a topic

bin/ka­fka­-to­ --create --zook­eeper localh­ost­:2181 --repl­ica­tio­n-f­actor 1 --part­itions 3 --topic test_topic

List topics

bin/ka­fka­-to­ --list --zook­eeper localh­ost­:2181

Push a file of messages to Kafka

bin/ka­fka­-co­nso­le-­pro­duc­ --brok­er-list localh­ost­:9092 --topic test_topic < file.log
# Messages should be one per line.

Listing messages from a topic

bin/ka­fka­-co­nso­le-­con­sum­ --zook­eeper localh­ost­:2181 --topic test_topic --from­-be­ginning

offset positions for consumer group per partition

bin/ka­fka­-co­nsu­mer­-of­fse­t-c­hec­ --zook­eeper localh­ost­:2181 --group {group-id} --topic {topic}
bin/ka­fka­-st­rea­ms-­app­lic­ati­on-­res­ --inpu­t-t­opics {topic} --appl­ica­tion-id {group-id} --boot­str­ap-­servers kafkah­ost­:9092 # To start over (reset offset to 0)