Configuring the eG Agent to Monitor Microsoft Azure Data Factory Using Azure ARM REST API

To configure the eG Agent to monitor Microsoft Azure Data Factory using Azure ARM REST API, you need to perform the following broad steps:

  • Register an Application with an Azure AD Tenant;

  • Determine the Tenant ID, the Application (Client) ID and Secret Key value associated with the registered Application;

  • Assign the Application to a Subscription and grant 'monitoring' rights to it.

If an Azure Subscription is already monitored in your environment, then you can use the same Application you created for Azure Subscription monitoring to monitor the Microsoft Azure Data Factory as well. In this case therefore, you do not need to perform any of the steps above.

On the other hand, if such an Application does not pre-exist, then proceed to create one. For that, you need to follow steps 1 - 3 above. The sub-sections below discuss each of these steps in detail.

Registering an Application with Azure Active Directory (AD)

An Azure AD Application is a digital identity and some associated configuration, which informs Azure AD about how to treat software which uses that digital identity.

The eG agent can pull performance metrics related to an Azure tenant, its services, and its resources, only if it communicates with an Azure AD tenant as an 'Application' with 'monitoring rights'.

If such an Application pre-exists with the target tenant, then you can configure the eG agent with the access credentials of that application. However, if no such application pre-exists, then first register a new Application with Azure AD and obtain the access tokens that Azure AD issues for that application.

To achieve this, do the following:

  1. Login to Microsoft Azure portal using https://portal.azure.com with valid credentials.

  2. Click on the Azure Active Directory indicated by Figure 115.

    Figure 115 : Selecting the Azure Active Directory option

  3. When Figure 116 appears, click on the App Registrations option in its left pane.

    Figure 116 : Clicking on the App Registrations option

  4. Figure 117 will then appear.

    Figure 117 : Registering new application

  5. To register a new application, click the New registration option indicated by Figure 117. The Register an application page will then appear as shown by Figure 118.

    Figure 118 : Specifying the details of the new application

  6. In Figure 118, specify the following:

    • The name of the application in the Name text box,

    • Select the type of the account from the Supported account types section.

      Supported account types Description

      Accounts in this organizational directory only

      Select this option if you want all user and guest accounts in your directory to use the application or API.

      Use this option if your target audience is internal to your organization.

      Accounts in any organizational directory

      Select this option if you want all users with a work or school account from Microsoft to use this application or API. This includes schools and businesses that use Office 365.

      Use this option if your target audience is business or educational customers and to enable multitenancy.

      Accounts in any organizational directory and personal Microsoft accounts

      Select this option if you want all users with a work or school, or personal Microsoft account to use your application or API. It includes schools and businesses that use Office 365 as well as personal accounts that are used to sign in to services like Xbox and Skype.

      Use this option to target the widest set of Microsoft identities and to enable multitenancy.

      Personal Microsoft Accounts only

      Select this option if you want the application or API to be used by only those users with personal accounts that are used to sign in to services like Xbox and Skype.
    • Then, enter the redirect URl (or reply URL) for your application in the Redirect URl text box. Typically, you need to provide the base URL of your app. For example, http://localhost:31544 might be the URL for a web app running on your local machine. Users would use this URL to sign in to a web client application. For public client applications, provide the URL used by Azure AD to return token responses. Enter a value specific to your application, such as https://DocApp.com//auth.

  7. Clicking the Register button in Figure 118 will create the Application. Then, Figure 119 will appear displaying the Essentials related to the new Application.

    Figure 119 : The successfully created application

  8. From the Essentials, you can obtain the Application ID and Directory ID (see Figure 119). Copy the Application ID and the Directory ID and paste them against the Client ID and TENANT ID text boxes while configuring eG tests for the target Azure component.

Obtaining the Client Secret

For the eG agent to obtain metrics from the target Microsoft Azure component, it is necessary to provide the client secret associated with the registered Application. For this, click on the Certificates & secrets option in the left pane of Figure 119. This will invoke Figure 120.

Figure 120 : Creating New Client Secret

Clicking on the New client secret button in the right panel of Figure 120 will invoke Figure 121. Specify the description of the client secret in the Description text box and choose an expiry period from the Expires section as shown in Figure 121.

Figure 121 : Adding the client secret

Clicking the Add button in Figure 121 will display a client secret value in the Value column of Figure 122.

Figure 122 : Generating the client secret value for the application

Note that the Value will disappear once you leave this page, so make sure that you copy the new client secret value in the clipboard by clicking the icon. Otherwise, you may need to generate a new client secret value. The client secret value has to be specified against the Client password field in the test configuration page.

Assigning the Application to a Subscription and Granting Monitoring Rights

Once the application is created, you need to grant that application monitoring access to all the resources contained with a specific subscription. To achieve this, do the following:

  1. In the Azure console, click on the Subscriptions option indicated by Figure 123.

    Figure 123 : Clicking on the Subscriptions option

  1. When Figure 124 appears, select the subscription that you want to access by clicking on it.

    Figure 124 : Selecting the subscription

  1. Figure 125 will then appear. Click the Access Control (IAM) option in the left panel of Figure 125.

    Figure 125 : Clicking on the Access Control (IAM) option

  1. The right panel will then change to display a Check Access section (see Figure 126). To grant monitoring rights for resources contained within a subscription, you need to create a new role. For that, click on the Add role assignment button in Figure 126.

    Figure 126 : Clicking on the Add role assignment button

  2. Figure 127 will then appear. Now, keep scrolling down the list of roles in Figure 127 until you find the Monitoring Reader role (see Figure 128). Select that role by clicking on it, and then click the Next button in Figure 128.

    Figure 127 : Scrolling down the list of roles that pre-exist

    Figure 128 : Selecting the Monitoring Reader role

  3. Figure 129 will then appear. Here, indicate which resource in the chosen subscription should be assigned the chosen role. For that, choose to Assign access to a User, group, or service principal, and then proceed to click on Select members.

    Figure 129 : Clicking on Select members

  4. Figure 130 will then appear. Using the Select members dialog box depicted by Figure 130, you have to select the application to which you want to assign the Monitoring Reader role. This should be the new application you previously registered with Azure AD. To search for this application, in the Select text box of Figure 130, enter the name of the application. Once the application is found, its name will appear below the Select text box. To select that application, click on it and then click on the Select button in Figure 130.

    Figure 130 : Selecting the applicaton registered with Azure AD

  5. Figure 131 will then appear, where your selection will be displayed. Click on the Next button in Figure 131 to continue.

    Figure 131 : A page indicating that the Monitoring Reader role has been assigned to the selected AD application

  6. This will invoke Figure 132. Quickly review the role assignment using Figure 132, and then click the Review + assign button to complete the process.

    Figure 132 : Review and assign the role to the chosen application

  7. Next, go back to the Figure 126 and click on the Add role assignment button therein, once again. When Figure 133 appears, type reader in the Search text box (i.e., the text box with the 'magnifying glass' icon within) to search for role names containing the string 'reader'. Once the matching role names are listed, scroll down the list to locate the Reader and Data Access role, and click on that role to select it.

    Figure 133 : Selecting the Reader and Data Access role

  8. Then, follow steps 6-9 above to assign the Reader and Data Access role to the new application you registered with Azure AD earlier.