
Introduction
Apache Kafka is amongst the fastest growing products out there that has been widely adopted by many companies across the globe. If you are using Kafka in production, is very important to be able to monitor and manage the cluster.
This article contains an updated list of the most popular and powerful monitoring tools for Apache Kafka clusters. You can also find an older list of recommendations that I proposed in an article written a few years back.
Specifically, in this article we will cover the following tools and services that can help you manage and monitor the health of Kafka clusters across organisations:
- Confluent Control Center and Health+
- Lenses
- Xinfra Monitor (formerly Kafka Monitor)
- Cruise Control
- CMAK (formerly Kafka Manager)
Confluent Control Center and Health+

The Confluent Control Center is a web-based tool that offers a user interface that facilitates cluster monitoring and administration. Specifically, you can quickly access an overview of the cluster health, access and observe messages, topics and Schema Registry services as well as executing ksql
queries.
Technically, the tool is compromised by components that collect metric data on producers and consumers, Kafka that is used to move the collected metrics and the Control Center application server that is used to analyse stream metrics.
Confluent Health + is another tool that can be used to to manage and monitor the Confluent Platform (it was actually introduced in Confluent Platform version 6.2).

Specifically, Health + can help you configure intelligent and rule-based alerts for which you can receive notifications on specific endpoints such as email or Slack. Additionally, you can view all the collected metrics as shown in the figure above as well as integrate Health+ with other monitoring tools such as Prometheus.
Lenses
Lenses offers a complete Kafka UI Tool that enables developers and engineers monitor the health Kafka infrastructure as well as application performance. Additionally, the tool can be used to configure alerts at real-time which facilitate incident management in an effective manner.

Lenses Tool also allows troubleshooting through an intuitive UI that lets you query Kafka using SQL.

You can see more details regarding Lenses’ pricing in their official website.
Xinfra Monitor (formerly Kafka Monitor)
Xinfra Monitor is an open-source tool developed by LinkedIn that is used to configure and execute long-running system tests over Kafka clusters. It helps developers and administrators capture bugs or regressions that are typically observed rarely or only after prolonged period of time.
Additionally, it helps you monitor the cluster using end-to-end pipelines that collect metrics such as end-to-end latency, service availability, producers and consumers availability, message loss rate and many more.
Cruise Control
Cruise Control, is another LinkedIn open-source tool that helps run Kafka clusters at large scale and is designed to address scalability issues such as broker death and workload balance.

Specifically, the tool allows you to monitor and track resource utilisation, query the cluster state and observe partitions, replicas and distribution. Additionally, you can monitor anomaly detection and configure alerts and self-healing processes as well as re-balancing.
Furthermore, you can also perform traditional administration such as adding, or removing brokers, adjusting replication factor, rebalancing the cluster, fixing offline replicas and triggering preferred leader election.

CMAK (formerly Kafka Manager)
CMAK (Cluster Manager for Apache Kafka) is an open-source tool that helps you manage Kafka clusters.
Specifically, the tool can help you manage various cluster, which is quite convenient if -say- you want to monitor clusters in different environments as shown below.

Additionally, CMAK can help you inspect the state of a cluster, including topics, consumers, offsets, brokers, replicas and partition distribution.

Furthermore, the tool can help you delete topics, reassign partitions, adjust partition number, update topic configuration and perform manual partition assignment through the interactive user interface.

The tool can be configure fairly easily – for more details about how to get it up and running, refer to the relevant documentation section.
Final Thoughts
In this article we covered the most commonly used monitoring tools that can be used to control and manage Apache Kafka clusters efficiently.
Personally, I believe that Confluent – the original creators of Kafka – offer the best tool in the market that also comes at certain cost. This means that there is a chance you may not be able to afford the licensing for this particular monitoring service and thus you need to consider other options covered in this article. Note that Health+ tool has a free tier – For more details regarding its pricing refer to the Confluent’s documentation.
The second best option is probably Lenses, that also requires a paid license but probably at a -significantly- lower cost. If you cannot afford licences, then LinkedIn Cruise Control and CMAK are good enough solutions that will definitely make your life easier when it comes to cluster management and monitoring.
Become a member and read every story on Medium. Your membership fee directly supports me and other writers you read.
You may also like