Monitoring AWS Managed Service Kafka
eG Enterprise offers a special-purpose monitoring model for the AWS Managed Service Kafka to monitor the status and overall performance of the target AWS Managed Service Kafka.
Figure 1 depicts the layer model of an AWS Managed Service Kafka.
Figure 1 : Layer model for AWS Managed Service Kafka
Every layer in the is mapped to various tests to determine the critical statistics related to the performance of the target AWS Managed Service Kafka. Using the metrics reported by the tests, administrators can find accurate answers for the following performance queries:
-
Does the target AWS MSK server uses all the physical, buffer and cached memory allocated to it?
-
Are the requests from producers, consumers and broker followers to the brokers addressed efficiently by the AWS Managed Service Kafka Message System without any communication lapses?
-
Is each partition managed by the controller active and running?
-
Are there any issues in the broker's connection to the consumer?
-
Is the request from consumer to broker and response from the broker to the consumer waits in the queue for a long time?
-
Is the CPU usage of each cluster in the target server too high leading to sudden spikes?
-
Is the request from consumer to follower and response from follower to consumer waits in the queue for a long time?
-
Are the network transmitting/receiving errors for the broker too high?
-
Is the request from consumer to producer and response from producer to consumer waits in the queue for a long time?
-
Is the average fetch throttle time taken for each request too high?
-
Is the message conversion and data transaction between the brokers stable enough for the target AWS Managed Service Kafka to communicate efficiently?
-
Is the health and availability status of each volume used by the EC2 instances in the monitored region sufficient?
-
Is the client’s connection to Zookeeper server stable?
-
Is the network connectivity for the broker stable enough for executing the Kafka operations?
