Pre-requisites for Monitoring Apache Impala

Each Apache Impala server has three daemon servers, namely - the Impala Daemon server, the Statestore Daemon server, and the Catalog Daemon server. Each of these daemon servers is responsible for performing critical server functions.

To obtain a holistic view of the performance of a target Apache Impala server, the eG agent connects to each of these daemon servers and collects statistics on their operations and health.

To allow the eG agent access to these daemon servers and their diagnostics via REST API, the following pre-requisites should be fulfilled:

Enabling the built-in web server - A built-in web server is included within each of the Impala daemons. The eG agent needs access to each of these web servers in order to run API calls on the daemons for metrics collection. Therefore, you need to enable the web server that is included with all the 3 daemon servers. Section Enabling Web server option in Cloudera Managerof this document explains how to achieve this using the Cloudera Manager.

Configuring the eG agent with 'secure' access to the built-in web server - Once you enable web server access to the eG agent, make sure that 'password authentication' is also enabled. This is needed, so that the eG agent is able to access the built-in web server for each daemon in a safe, secure manner. To know how to use the Cloudera manager and enable 'password authentication' for every daemon server, refer to Section Configuring Username and Password for Web server of this document. After you set exclusive access credentials for each daemon server in Cloudera, make sure you configure the eG tests with these credentials.

Configuring the eG agent with the API Endpoint URLs - Each daemon server has a different endpoint URL. You need to configure the eG agent with the endpoint URL of each daemon, so that the agent can access the URL and pull metrics on the health of the target daemon servers.

Each URL contains the IP address of the target daemon and the port at which the daemon listens for requests from the eG agent. By default, the daemons listen on the following ports:

  • Impala Daemon Port: 25000

  • Statestore Daemon Port: 25010

  • Catalog Daemon Port: 25020

If say, the IP address of the Impala daemon server is 172.16.14.164, then you need to configure the Impalad URL parameter of the eG tests with the value http://172.16.14.164:25000. Similarly, you should configure the values of the Statestored and Catalogd parameters of the eG tests.

Sometimes, a daemon server can have multiple IP addresses. In this case, the corresponding URL parameter should be configured with a comma-separated list of endpoint URLs. For example, if the Impala daemon server supports multiple IP addresses, then your Impalad URL specification can be : http://172.16.14.165:25000, http://172.16.14.165:25000.

Enabling Web server option in Cloudera Manager

For collecting the metrics, we require Cloudera Manager to enable the Web servers for the three Daemon servers and it is achieved enabling the Enable Web Servers for Web UI option for each Daemon server in Cloudera Manager (see Figure 1). The following steps shows on how to enable Enable Web Servers for Web UI option for all the three Daemon servers.

  • Login into the Cloudera Manager.

  • To configure the settings for Impala Daemon server,

    • Navigate to Clusters > Impala Service > Configuration from Cloudera manager.

    • Select Scope > Impala Daemon.

    • Search for "enable_webserver" using the Search box in the Configuration tab.

    • Click the check box for enable Impala Daemon Web Server

    • Click Save Changes and restart the Impala service.

  • To configure the settings for Impala Statestore server,

    • Navigate to Clusters > Impala Service > Configuration from Cloudera manager

    • Select Scope > Impala Statestore

    • Search for "enable_webserver" using the Search box in the Configuration tab.

    • Click the check box for enable Statestore Daemon Web Server.

    • Click Save Changes and restart the Impala service.

  • To configure the settings for Impala Catalog server,

    • Navigate to Clusters > Impala Service > Configuration from Cloudera manager

    • Select Scope > Impala Catalog server

    • Search for "enable_webserver" using the Search box in the Configuration tab.

    • Click the check box for enable Catalog Daemon Web Server.

    • Click Save Changes and restart the Impala service.

Figure 1 : Enabling Impala Web Servers in Web UI in Cloudera Manager

Configuring Username and Password for Web server

To access the Web server of each Daemon server for collecting the metrics, the username and password authentication is required and it is generated in Cloudera Manager. For our example, the steps involved for configuring the username and password for Catalog daemon server is listed below (see Figure 2). Similar procedure is followed for Impala Daemon and Statestore Daemon servers.

  • Navigate to Clusters > Impala Service > Configuration from Cloudera manager.

  • Search for "password" using the Search box in the Configuration tab. This should display the password-related properties (Username and Password properties) for the Impala Daemon, StateStore, and Catalog Server. If there are multiple role groups configured for Impala Daemon instances, then the search should display all of them.

  • Enter a username and password into these fields.

  • Click Save Changes and restart the Impala service.

    Figure 2 : Username and password authentication for Cloudera Impala

After configuring the above settings (see Figure 1), you will be able to access the Web server for the Catalog Daemon using the configured username and password entries.

In the above steps, we have shown the prerequisites for monitoring the Apache Impala server in Cloudera Manager. For other vendors, please check with the respective vendor administrators on how to enable Impala Web server in their environment and how to configure password protection for accessing the Web server in their environment.

Note:

If username and password are configured in Cloudera Manager, then specify the same set of values for both User and Password parameters in eG Test Configuration page similar to what we have configured as outlined in Configuring Username and Password for Web server of this document.