KafkaCat Cheat Sheet: Essential Commands for every Kafka Developers

KafkaCat is a powerful command-line utility designed to help you interact with Kafka topics, produce and consume messages, and navigate Kafka clusters seamlessly. Whether you’re a seasoned Kafka expert or just getting started, having a handy cheat sheet of KafkaCat commands can significantly enhance your productivity.

Installation

Before you dive into the world of KafkaCat, you need to have it installed on your system. Here’s how to set it up:

Linux (using apt)

Open your terminal and execute the following commands:

sudo apt-get update
sudo apt-get install kafkacat

macOS (using Homebrew)

If you’re on macOS and have Homebrew installed, installing KafkaCat is a breeze. Open your terminal and run:

brew install kafkacat

Other Platforms

For other platforms, you can compile KafkaCat from source using the following steps:

1. Clone the KafkaCat repository:

git clone https://github.com/edenhill/kafkacat.git

2. Navigate to the KafkaCat directory:

cd kafkacat

3. Build and install KafkaCat:

./configure 
make 
sudo make install

Commands:

Basic Usage

1. Produce a Message :

kafkacat -b <broker> -t <topic> -P
-b: Specify Kafka broker(s).
-t: Specify the target topic.
-P: Enter the producer mode.

2. Consume Messages :

kafkacat -b <broker> -t <topic> -C
-b: Specify Kafka broker(s).
-t: Specify the target topic.
-C: Enter the consumer mode.

3. Consume from a Specific Offset :

kafkacat -b <broker> -t <topic> -C -o <offset>
-o: Specify the offset to start consuming from.

Message Manipulation

1. Copy Messages from One Topic to Another

kafkacat -b <broker> -t <source_topic> -C | kafkacat -b <broker> -t <destination_topic> -P

2. Finding Messages :

kafkacat -b <broker> -t <topic> -C -o <offset> -c <message_count> -f <format_string>
  
-c: Specify the number of messages to consume.
-f: Define a custom format string to display messages.

Advanced Operations

1. Consume from the Latest Offset

kafkacat -b <broker> -t <topic> -C -o -1
-o -1: Consume from the latest offset.

2. Consume from the Earliest Offset

kafkacat -b <broker> -t <topic> -C -o -2
-o -2: Consume from the earliest offset.

3. Setting Consumer Group and Client ID

kafkacat -b <broker> -t <topic> -C -G <consumer_group> -C -C <client_id>
-G: Specify the consumer group.
-C: Specify the client ID.

Note : If security is enabled, each kafkacat command should include security parameters like

kafkacat -b <broker> -t <topic> -C -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X sasl.username=<your_username> -X sasl.password=<your_password> --consumer.config <path_to_config_file>

-b: Specify Kafka broker(s).
-t: Specify the topic to consume from.
-C: Enter consumer mode.
-X security.protocol=SASL_SSL: Set the security protocol to SASL_SSL.
-X sasl.mechanisms=PLAIN: Specify the SASL mechanism.
-X sasl.username=<your_username>: Provide your SASL username.
-X sasl.password=<your_password>: Provide your SASL password.
--consumer.config <path_to_config_file>: Specify the path to a consumer 

In Conclusion

As you conclude your exploration of KafkaCat commands with this cheat sheet, it’s important to remember that the examples provided are just the tip of the iceberg. KafkaCat offers a wealth of options and configurations for each command, allowing you to fine-tune your interactions with Kafka clusters to meet your specific needs.

Exploring the documentation and running kafkacat --help unveils a wealth of features to suit diverse scenarios. Whether you're a seasoned developer or a newcomer, KafkaCat enhances Kafka cluster engagement. This cheat sheet serves as a quick reference, encouraging you to discover KafkaCat's potential and flexibility in your Kafka endeavors.

Looking for a specific command? Feel free to ask in the comments.

Keep Learning !