This is a draft cheat sheet. It is a work in progress and is not finished yet.
Describe configs for a topic
bin/kafka-configs.sh --zookeeper localhost:2181 --describe --entity-type topics --entity-name test_topic |
Set retention times
bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic test_topic --config retention.ms=1000 # Deprecated way |
bin/kafka-configs.sh --zookeeper localhost:2181 --alter --entity-type topics --entity-name test_topic --add-config retention.ms=1000 # Modern way |
If you need to delete all messages in topic, you can exploit the retention times. First set the retention time to something very low (1000 ms), wait a few seconds, then revert the retention times back to the previous value.
Note: The default retention time is 24 hours (86400000 millis).
Delete a topic
bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic test_topic |
Describe a topic
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test_topic |
Add a partition
bin/kafka-topics.sh --alter --zookeeper localhost:2181 --topic test_topic --partitions 3 |
Create a topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic test_topic |
List topics
bin/kafka-topics.sh --list --zookeeper localhost:2181 |
Push a file of messages to Kafka
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test_topic < file.log |
# Messages should be one per line. |
Listing messages from a topic
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test_topic --from-beginning |
offset positions for consumer group per partition
bin/kafka-consumer-offset-checker.sh --zookeeper localhost:2181 --group {group-id} --topic {topic} |
bin/kafka-streams-application-reset.sh --input-topics {topic} --application-id {group-id} --bootstrap-servers kafkahost:9092 # To start over (reset offset to 0) |
|
