Azure ADF Pipeline Runs Test

In Azure Data Factory (ADF), a pipeline run refers to an execution instance of a pipeline. When a pipeline is triggered either manually, on a schedule, or in response to an event, a pipeline run is created. This test monitors those pipeline runs and alerts the administrators on the maximum pipeline run failures and queued pipeline runs that could affect the execution instance of a pipeline.

Target of the test : Microsoft Azure Data Factory

Agent deploying the test : An external agent

Outputs of the test : One set of results for each Resource Group : Pipeline in the target Microsoft Azure Data Factory.

Configurable parameters for the test
Parameters	Description
Test Period	How often should the test be executed.
Host	The host for which the test is to be configured.
Subscription ID	Specify the GUID which uniquely identifies the Microsoft Azure Subscription to be monitored. To know the ID that maps to the target subscription, do the following: Login to the Microsoft Azure Portal. When the portal opens, click on the Subscriptions option (as indicated by Figure 1). Figure 1 : Clicking on the Subscriptions option Figure 2 that appears next will list all the subscriptions that have been configured for the target Azure AD tenant. Locate the subscription that is being monitored in the list, and check the value displayed for that subscription in the Subscription ID column. Figure 2 : Determining the Subscription ID Copy the Subscription ID in Figure 2 to the text box corresponding to the SUBSCRIPTION ID parameter in the test configuration page.
Tenant ID	Specify the Directory ID of the Azure AD tenant to which the target subscription belongs. To know how to determine the Directory ID, refer to Configuring the eG Agent to Monitor Microsoft Azure Data Factory Using Azure ARM REST API
Client ID, Client Password, and Confirm Password	To connect to the target subscription, the eG agent requires an Access token in the form of an Application ID and the client secret value. For this purpose, you should register a new application with the Azure AD tenant. To know how to create such an application and determine its Application ID and client secret, refer to Configuring the eG Agent to Monitor Microsoft Azure Data Factory Using Azure ARM REST API. Specify the Application ID of the created Application in the Client ID text box and the client secret value in the Client Password text box. Confirm the Client Password by retyping it in the Confirm Password text box.
Proxy Host	In some environments, all communication with the Azure cloud could be routed through a proxy server. In such environments, you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics. To enable metrics collection via a proxy, specify the IP address of the proxy server and the port at which the server listens against the Proxy Host and Proxy Port parameters. By default, these parameters are set to none, indicating that the eG agent is not configured to communicate via a proxy, by default.
Proxy Username, Proxy Password and Confirm Password	If the proxy server requires authentication, then, specify a valid proxy user name and password in the Proxy Username and Proxy Password parameters, respectively. Then, confirm the password by retyping it in the Confirm Password text box.
DD Frequency	Refers to the frequency with which detailed diagnosis measures are to be generated for this test. The default is 1:1. This indicates that, by default, detailed measures will be generated every time this test runs, and also every time the test detects a problem. You can modify this frequency, if you so desire. Also, if you intend to disable the detailed diagnosis capability for this test, you can do so by specifying none against DD Frequency.
Detailed Diagnosis	To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option. The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled: The eG manager license should allow the detailed diagnosis capability Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.

Measurements made by the test
Measurement	Description	Measurement Unit
Queued pipeline runs	Indicates the number of pipeline runs that were in queue in this pipeline.	Number
In progress runs	Indicates the number of pipeline runs that were in progress in this pipeline.	Number
Succeeded pipeline runs	Indicates the number of pipeline runs that succeeded in this pipeline.	Number
Failed pipeline runs	Indicates the number of pipeline runs that failed in this pipeline.	Number
Cancelling pipeline runs	Indicates the number of cancelling pipeline runs in this pipeline.	Number
Cancelled pipeline runs	Indicates the number of pipeline runs that were cancelled in this pipeline.	Number
Long running pipeline runs	Indicates the number of pipeline runs that were long running in this pipeline.	Number
Total pipeline runs	Indicates the total number of pipeline runs in this pipeline.	Number
Availability	Indicates the percentage of pipeline runs that were available in this pipeline.	Percent
Failed runs rate	Indicates the rate at which the pipeline runs failed in this pipeline.	Percent