Apache Kafka Broker layer

Using the tests associated with the Apache Kafka Broker layer (see Figure 1), you can monitor the following:

• Rate at which data sent from producers is consumed by the broker.

• Rate at which individual messages are consumed by the broker.

• Fetch request rate for requests that have failed.

• Rate at which data is sent from one broker to other brokers.

• Rate at which data sent from other brokers is consumed by the follower broker.

• Outgoing/Incoming byte rate of reassignment traffic.

• Number of brokers designated as controllers.

• Number of partitions without an active leader and to check whether/not they are writable or readable.

• Imbalance count in the preferred replica.

• Number of coordinator groups currently in stable/unstable state.

• Number of consumer groups that are currently in preparing/completing re-balance state.

• Number of offline log directories on a live broker.

• Size of the request/response queues.

• Total number of disconnections across all processors.

• Number of replicas for which this broker is the leader.

• Number of offline replicas on a live broker.

• Number of reassigning leader partitions on a broker.

• Number of partitions that have not been fully replicated in the follower replicas.

• Number of partitions under the minimum In-Sync Replica (ISR) count.

• Size of Fetch, Produce and Request queues.

• Status of the broker’s connection to Zookeeper.

• Number of zookeeper clients that are connected to the ensemble and ready to execute operations.

• Number of attempts to connect the ensemble but failed due to incorrect credentials.

Figure 1 : The list of tests associated with the Apache Kafka Broker layer