Apache Web Transactions By Site Test

In the digital world, enterprises often end up paying a very high price for not paying enough attention to the end-user experience with their business critical web sites/applications. Unhappy users can cause businesses to lose millions by way of revenue, support costs, and reputation! To make users happy, administrators should closely track transactions to web sites/applications hosted on their production servers, rapidly capture 'slow' or 'error-prone' transactions, and resolve the bottleneck before users complain.The Apache Web Transactions By Site test helps administrators achieve this!

This test scans the Access log of the Apache web server and pulls metrics on requests processed from it. With the help of these metrics, administrators can continuously assess the processing ability of the server, and proactively detect potential latencies in processing. Detailed diagnostics provided by the test point administrators to the precise business transactions that are slow and the users who may be impacted by the slowness, so that remedial measures can be initiated even before users notice the slowness.

In addition, the test also reveals the number and type of HTTP error codes that were returned in response to business transaction requests, This helps administrators understand why the errors occurred. Deep-dive insights on these error codes are also available. These advanced analytics pinpoint the exact business transactions that encountered these errors and the users who were impacted. Using this information, administrators can fix the errors, well before users complain.

Note:

This test will be able to 'read' from the Access log and report metrics, only if the eG agent install user is vested with 'read' permission to the log file.

Target of the test : An Apache web server

Agent deploying the test : An internal agent

Outputs of the test : One set of results for the Apache web server being monitored

Configurable parameters for the test

Parameter

Description

Test Period

How often should the test be executed.

Host

The host for which the test is to be configured.

Port

The port to which the specified Host listens. By default, this is set to 80.

Access Log Path

This test collects the required metrics by reading the Access log on the target Apache Web server. This log is responsible for recording data for all requests processed by the Apache server. To enable the test to read from the Access log, you need to specify the full path to the access log file in this text box. By default, this parameter is set to default. This means that typically, the test looks for the Access log in its default path - this is "/var/log/httpd"on Unix, and <APACHE_INSTALL_DIR>\logs on Windows. If the Access log is in a different location in your environment, then you have to override this default setting.

Log Format

The Log Format parameter is where you need to define what type of details should be logged in the Access log. By default, this parameter is set to default. In this case therefore, the test uses the following format as the default format on Unix: %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\". On Windows, the following format is used as the default format: %h %l %u %t \"%r\" %>s %b.

If you want to configure a custom log format, then override this default setting by removing the percentage directives that you do not want from the format strings above. Before you attempt to modify the log format, use the table below to understand what each of the percentage directives represent:

Directive	Description
%h	IP address of the client (remote host) which made the request to the server
%l	The RFC 1413 identity of the client determined by 'identd' on the clients machine
%u	Userid of the person requesting the document as determined by HTTP authentication
%t	The time that the request was received
\"%r\"	The request line from the client
(%>s	The status code that the server sends back to the client
%b	The size of the object returned to the client, not including the response headers
\"%{Referer}i\"	The "Referer" (sic) HTTP request header. This gives the site that the client reports have been referred from.
\"%{User-agent}i\"	The User-Agent HTTP request header. This is the identifying information that the client browser reports about itself.

Exclude URL Pattern

Specify a comma-separated list of transaction URL patterns that need to be excluded from the scope of monitoring - for example, */shopping/hardware*,*/shopping/tiles*. By default, this parameter is set to none, indicating that all transactions are monitored by default.

Max Requests in DD

By default, this parameter is set to 1000. This indicates that detailed diagnosis of the Total requests processed and the Requests processed successfully measures will report detailed metrics for a maximum of 1000 requests in a chosen measure period. If you want to increase this limit, then increase the value of this parameter. Since any increase in the value of this parameter can increase the load on the eG database, we recommend that you tune and right-size your eG database before reconfiguring this parameter.

Show Total Requests DD

By default, this flag is set to No. This implies that by default, detailed metrics will not be available for the Total requests processed measure of this test. To enable detailed diagnosis for this measure, you can set this flag to Yes. However, before doing so, make sure that your eG database is well-tuned and is sufficiently sized.

Show Success Requests DD

By default, this flag is set to No. This implies that by default, detailed metrics will not be available for the Requests processed successfully measure of this test. To enable detailed diagnosis for this measure, you can set this flag to Yes. However, before doing so, make sure that your eG database is well-tuned and is sufficiently sized.

Use Sudo

By default, the Use Sudo parameter is set to No. This indicates that, by default, the eG agent install user will be able to read the access log file and collect the required metrics. If the access log file could not be read by the eG agent install user, then, set this flag to Yes. This test will then report the metrics by executing the usr/bin/sudo command or /usr/sbin/sudo command.

However, in some highly secure environments, the eG agent install user may not have the permissions to execute this command directly. In such cases, do the following:

Edit the sudoers file on the target host and append an entry of the following format to it:

For instance, if the eG agent install user is eguser, then the entries in the sudoers file should be:

eguser ALL=(ALL) NOPASSWD: /usr/bin/sudo

eguser ALL=(ALL) NOPASSWD: /usr/sbin/sudo
Finally, save the file.
Then, when configuring the test using the eG admin interface, se tthis parameter to Yes. This will enable the eG agent to use sudo to collect the required metrics from the access log file.

Sudo Path

By default, this parameter is set to none. This indicates that the eG agent would automatically discover the default location of the sudo command i.e. /usr/bin or /usr/sbin, when the Use Sudo flag is set to Yes. However, if the sudo command is installed in a different location in your environment, then specify that location in the Sudo Path text box.

DD Frequency

Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 1:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD frequency.

Detailed Diagnosis

To make diagnosis more efficient and accurate, the eG suite embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option.

The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:

The eG manager license should allow the detailed diagnosis capability
Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.

Measurements made by the test
Measurement	Description	Measurement Unit	Interpretation
Requests	Indicates the rate of requests to the web sites on the server.	Requests/Sec
Errors	Indicates the percentage of error requests to the web sites on the web server.	Percent	Ideally, the value of this measure should be 0.
Aborts	Indicates the percentage of requests to web sites on this server, which were aborted .	Percent	Ideally, the value of this measure should be 0.
Data transmitted	Indicates the rate at which the data is transmitted by the server.	KB/sec
Avg response time	Indicates the average time taken by the web sites on the server to respond to transaction requests.	Seconds	A high value is indicative that many transactions are slow. To identify the slow transactions, use the detailed diagnosis of the Total requests processed measure or Requests processed successfully measure (if enabled).
Total requests processed	Indicates the total number of transaction requests that are currently processed by the web sites on the server.	Number	This measure is a good indicator of current load on the server. The detailed diagnosis of this measure, if enabled, provides in-depth insights into the requests processed. The details include the URL of each transaction request, the user who initiated the transaction, the HTTP response status for the transaction, the time taken by the server to process the transaction, the throughput per transaction, the protocol used, and more. Using these useful metrics, administrators can precisely pinpoint the transaction that took the longest to be processed, and who initiated it.
Requests processed successfully	Indicates the number of requests that are successfully processed by the web sites on the server.	Number	Ideally, the value of this measure should be equal to or close to the value of the Total requests processed measure. A value that is much lower than that of the Total requests processed measure could mean that many requests failed processing or are pending processing. Either way, it is not a sign of good health. The detailed diagnosis of this measure, if enabled, provides in-depth insights into the successful requests. The details include the URL of each transaction request, the user who initiated the transaction, the HTTP response status for the transaction, the time taken by the server to process the transaction, the throughput per transaction, the protocol used, and more.
Total redirection requests	Indicates the total number of HTTP 3xx responses returned by the web sites on the server.	Number	A site returns 300 series response codes when it redirects requests to other pages. Redirection is performed for various reasons. For example, when the client browser may have to request a different page on the server or to repeat the request by using a proxy server. These codes helps the client to ensure that a redirection to a different resource or URL should take place to complete the requests and access the desired resource. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 3xx responses, and which users were impacted.
Moved permanently	Indicates the number of HTTP 301 responses returned by the web sites on the server.	Number	An HTTP 301 response is returned if the URL of the requested resource has been changed permanently. The new URL is given in the response. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 301 responses, and which users were impacted.
Object moved	Indicates the number of HTTP 302 responses returned by the web sites on the server.	Number	A 302 Found message is an HTTP response status code indicating that the requested resource has been temporarily moved to a different URL. Since the location or current redirection directive might be changed in the future, a client that receives a 302 response code should continue to use the original URL for future requests. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 302 responses, and which users were impacted.
Not modified	Indicates the number of HTTP 304 responses returned by the web sites on the server.	Number	The HTTP 304 response code indicates that there is no need to retransmit the requested resources. It is an implicit redirection to a cached resource. This means that the requested resource is already in the cache and the resource has not been modified since it was cached. Therefore, the client can use the cached copy of the resource, instead of downloading it from the server. This happens when the request method is safe, like a GET or a HEAD request, or when the request is conditional and uses a If-None-Match or a If-Modified-Since header. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 304 responses, and which users were impacted.
Temporary redirect	Indicates the number of HTTP 307 responses returned by the web sites on the server.	Number	The HTTP 307 Temporary Redirect status response code indicates that the resource requested has been temporarily moved to the URL given by the Location headers. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 307 responses, and which users were impacted.
Total 4xx requests	Represents the total number of HTTP 4xx (client error) responses returned by the web sites on the server.	Number	The 4xx HTTP status codes indicate that an error occurred and the client browser appears to be at fault. For example, the client browser may have requested a page that does not exist or may not have provided valid authentication information. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 4xx responses, and which users were impacted.
Bad requests	Indicates the number of HTTP 400 (bad requests) responses returned by the web sites on the server.	Number	The web site returns the HTTP 400 code when the request could not be understood by the server due to malformed syntax. This implies that the client should not repeat the request without modifications. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 400 responses, and which users were impacted.
Total access denied requests	Indicates the number of HTTP 401 responses returned by the web sites on the server.	Number	The HyperText Transfer Protocol (HTTP) 401 Unauthorized response status code indicates that the client request has not been completed because it lacks valid authentication credentials for the requested resource. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 401 responses, and which users were impacted.
Logon failed	Indicates the number of requests that are currently denied access resources due to login failure.	Number	The login failure error occurs when the logon attempt is unsuccessful, probably because of a user name or password that is not valid. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 401.1 responses, and which users were impacted.
Logon failed due to server configuration	Indicates the number of login attempts to the web sites on the server that failed due to lack of proper server configuration.	Number	Ideally, the value of this measure should be zero. The HTTP 401.2 status code indicates a problem in the authentication configuration settings on the server. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 401.2 responses, and which users were impacted.
Authorization failed requests	Indicates the number of requests to the web sites on the server for which authorization failed.	Number	Ideally, the value of this measure should be zero. Authorization errors will be reported when the requests are not authenticated by ACL on resource or filter or ISAPI/CGI application. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 401.3, 401.4, and 401.5 responses, and which users were impacted.
Forbidden errors	Indicates the number of HTTP 403 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. A web site encounters forbidden errors for various reasons, for example, when the web site receives too many requests from the same client. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 403 responses, and which users were impacted.
Not found errors	Indicates the number of HTTP 404 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. The 404 error is reported when the server has not found anything matching the Request-URL. This error code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 404 responses, and which users were impacted.
Method not allowed	Indicates the number of HTTP 405 responses returned by the web sites on the server.	Number	The 405 Method Not Allowed response code indicates that the method specified in the Request-Line is known by the origin server but is not supported by the target resource. To avoid this, administrators should include the response MUST an Allow header containing a list of valid methods for the requested resource. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 405 responses, and which users were impacted.
Client browser not accept the MIME type	Indicates the number of HTTP 406 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. The HTTP 406 (Not Acceptable client) error code indicates that the server cannot produce a response matching the list of acceptable values defined in the request's proactive content negotiation headers, and that the server is unwilling to supply a default representation. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 406 responses, and which users were impacted.
Request timed out	Indicates the number of HTTP 408 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. The 408 Request Timeout error indicates that the request sent to the web server (e.g. a request to load a web page) took longer than the web server was prepared to wait. In other words, your connection with the web site "timed out". Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 408 responses, and which users were impacted.
Precondition failed	Indicates the number of HTTP 412 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. The precondition given in one or more of the request-header fields evaluated to false when it was tested on the server. This response code allows the client to place preconditions on the current resource metainformation (header field data) and thus prevent the requested method from being applied to a resource other than the one intended. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 412 responses, and which users were impacted.
Total internal server errors	Indicates the total number of HTTP response codes in the range of 500-511.	Number	Ideally, the value of this measure should be zero. Response codes in the range of 500-511 often indicate server errors. The server error messages are reported for a wide variety of server-side errors. For more information on the errors, you can refer event viewer logs and find out the reason why the errors occur. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 5xx responses, and which users were impacted.
Module or ISAPI errors	Indicates the number of HTTP 500.0 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. The 500.0 error occurs when the ISAPIModule module is missing from the modules list for the web site. ISAPI filters (modules) are libraries loaded by the IIS web server. Every incoming request and outgoing response passes through the filters, and they are free to perform any handling or translation they wish. They can be used for authentication, content transformation, logging, compression, and myriads of other uses. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 500.0 responses, and which users were impacted.
Application is shutting down errors	Indicates the number of HTTP 500.11 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. A web site reports 500.11 HTTP error code when an application on the server is shutting down, and thus the received request cannot be processed by the web site. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 500.11 responses, and which users were impacted.
Application is busy restarting errors	Indicates the number of HTTP 500.12 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. This error occurs when you tried to load an ASP page while IIS server was in the process of restarting the application. This error message will disappear when you refresh the page. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 500.12 responses, and which users were impacted.
Web server busy errors	Indicates the number of HTTP 500.13 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 500.13 responses, and which users were impacted.
Internal ASP errors	Indicates the number of HTTP 500.100 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. These errors occur when a dynamic-link library (DLL) that is required by the Microsoft Data Access Components is not registered. To alleviate this kind of errors, administrators should register the DLL before the errors cause serious impacts on the key transactions. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 500.100 responses, and which users were impacted.
Service unavailable errors	Indicates the number of HTTP 503 responses returned by the web sites on the server.	Number	Ideally, the value of this measure should be zero. The 503 errors occur when the application pool/service is stopped or mismatch in user identity settings is recorded. Administrators can fix these issues by restarting the stopped application pool/service and updating the user account settings. Use the detailed diagnosis of this measure to know which transaction requests returned HTTP 503 responses, and which users were impacted.
Remote hosts	Indicates the number of remote hosts that are currently accessing the web sites on the server.	Number	Use the detailed diagnosis of this measure to know which remote hosts are accessing the web sites on the server.