Azure Provisioning Logs Test

Azure AD application provisioning refers to automatically creating user identities and roles in the cloud applications that users need access to. In addition to creating user identities, automatic provisioning includes the maintenance and removal of user identities as status or roles change. Common scenarios include provisioning an Azure AD user into SaaS applications like Dropbox, Salesforce, ServiceNow, and more.

Azure AD also supports provisioning users into applications hosted on-premises or in a virtual machine, without having to open up any firewalls.

Additionally, inbound provisioning from HCM (Human Capital Management) applications (like WorkDay) to Azure AD and Active Directory is also supported.

Figure 1 depicts how fthe Azure AD application provisioning service performs outbound provisioning to SaaS applications.

Figure 1 : Outbound user provisioning from Azure AD to SaaS applications

Figure 2 describes how the provisioning service supports inbound provisioning.

Figure 2 : Inbound user provisioning from HCM to Azure AD and Windows Server Active Directory

The steps below describe how the provisioning service performs automatic provisioning:

The service first attempts to authorize access to the target application by making a request for a user.
Next, the provisioning service retrieves/imports the user from the source system. The user attributes that the service retrieves are used later to:
- Evaluate whether the user is in scope for provisioning.
- Check the target system for an existing user.
- Determine what user attributes to export to the target system.
Then, the provisioning service determines whether the user is in scope for provisioning.
The service then attempts to match the user that was retrieved in the step 2 above with a user in the target system.
Finally, the provisioning service takes an action, such as creating, updating, deleting, or skipping the user.

If errors/failures occur in any of the above-mentioned steps, then it will cause the entire provisiong automation to fail. Consequently, administrators may be forced to manually manage user identities in SaaS applications. This in turn can be cumbersome. To avoid this, administrators should be able to track the progress of provisioning events, rapidly detect provisioning failures, isolate why provisioning failed and at which step, and fix it. This is where the Azure Provisioning Logs test helps!

Typically, all operations run by the user provisioning service are recorded in the Azure AD provisioning logs. This includes all read and write operations made to the source and target systems, and the user data that was read or written during each operation. The status of each operation, errors encountered (if any), and reasons for the same, are also recorded in these logs. The Azure Provisioning Logs test scans these provisioning logs for the status of provisioning operations, and alerts administrators to those operations that have failed or may potentially fail. Detailed diagnostics reveal which operations failed/may fail, when, at what step, and why. With the help of this information, administrators can quickly and efficiently troubleshoot the failures and warnings, and ensure that the quality of the provisioning service does not deteriorate. Additionally, the test also reveals the following:

Are too many provisioning operations failing when they are attempting a specific action - eg., Create, Update, Delete etc.?
Are provisioning operations failing too frequently at a specific step?

These analytics will help administrators unearth underlying configuration issues, which will have to be addressed for these disturbing error patterns to dissappear.

Also, using the detailed metrics provided by this step,administrators can also identify those operations that have spent too much time at a particular step of the provisioning process, and investigate the reasons for the slowness.

All operations run by the user provisioning service are recorded in the Azure AD provisioning logs. This includes all read and write operations made to the source and target systems, and the user data that was read or written during each operation.

Target of the Test: A Microsoft Azure Active Directory

Agent deploying the test: A remote agent

Output of the test: One set of results for the Azure Active Directory tenant being monitored

Configurable parameters for the test
Parameters	Description
Test Period	How often should the test be executed.
Host	The host for which the test is to be configured.
Tenant ID	Specify the Directory ID of the Azure AD tenant to which the target subscription belongs. To know how to determine the Directory ID, refer to Configuring the eG Agent to Monitor Microsoft Azure Active Directory Using Microsoft Graph API
Client ID, Client Password, and Confirm Password	To connect to Azure AD, the eG agent requires an Access token in the form of an Application ID and the client secret value. If a Microsoft Azure Subscription component is already monitored in your environment, then you would have already created an Application for monitoring purposes. You can provide the Application ID and Client Secret value of that application here. However, if no such application pre-exists, you will have to create one for monitoring Azure AD. To know how to create such an application and determine its Application ID and Client Secret, refer to Configuring the eG Agent to Monitor Microsoft Azure Active Directory Using Microsoft Graph API. Specify the Application ID of the Application in the Client ID text box and the client secret value in the Client Password text box. Confirm the Client Password by retyping it in the Confirm Password text box.
Proxy Host and Proxy Port	In some environments, all communication with the Azure cloud be routed through a proxy server. In such environments, you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics. To enable metrics collection via a proxy, specify the IP address of the proxy server and the port at which the server listens against the Proxy Host and Proxy Port parameters. By default, these parameters are set to none, indicating that the eG agent is not configured to communicate via a proxy, by default.
Proxy Username, Proxy Password and Confirm Password	If the proxy server requires authentication, then, specify a valid proxy user name and password in the Proxy Username and Proxy Password parameters, respectively. Then, confirm the password by retyping it in the Confirm Password text box.
Provisioning Log Workspace Name	Typically, the Provisioning logs are sent to a Log Analytics Workspace. By default, the Log Analytics Workspace Name parameter is set to All. his indicates that the test reads sign-in data from all Log Analytics Workspaces configured for the target tenant, by default. However, if you want the test to use only specific Log Analytics Workspaces for metrics collection, then provide the names of these workspaces here as a comma-separated list. To determine the names of the workspaces, do the following: Login to the Microsoft Azure Portal and select the Sign-in Logs option (see Figure 3). Figure 3 : Selecting the Sign-in Logs option Figure 4 will then appear listing the log entries. Next, click on the Export Data Settings button indicated by Figure 4. Figure 4 : Clicking on the Export Data Settings button The diagnostic settings that pre-exist for the sign-in logs will then appear. If any of the existing diagnostic settings have already been configured with Log Analytics Workspaces, then the Log Analytics workspace column highlighted in Figure 5 will display these workspace names. You can configure the LOG ANALYTICS WORKSPACE NAME parameter of this test with any of these workspace names. If required, you can even configure this parameter with two/more workspaces listed in Figure 5, as a comma-separated list. Figure 5 : List of Log analytics workspaces However, If the Log Analytics workspace column in Figure 5 is blank for all the existing diagnostic settings, it is a clear indication that the sign-in logs are not yet configured to be sent to any Log Analytics Workspace. In this case therefore, you should create a new diagnostic setting, where a Log Analytics Workspace is configured as the destination for the Sign-in logs. To achieve this, follow the procedure detailed in Configuring the Sign-in Logs to be Sent to a Log Analytics Workspace.
Detailed Diagnosis	To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option. The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled: The eG manager license should allow the detailed diagnosis capability Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0.

Measures made by the test:
Measurement	Description	Measurement Unit	Interpretation
Total provisioning events	Indicates the total number of provisioning operations performed by the Azure AD provisioning service.	Number
Successful events	Indicates the number of provisioning operations that successfully provisioned users to applications.	Number	Use the detailed diagnosis of this measure to know which provisioning operations were successful.
Failed events	Indicates the number of provisioning operations that failed.	Number	Ideally, the value of this measure should be 0. If this measure reports a non-zero value, then use the detailed diagnosis of this measure to know which operations failed, when, why, and at which step.
Warned events	Indicates the number of provisioning operations that are in the Warning state.	Number	Ideally, the value of this measure should be 0. If this measure reports a non-zero value, then use the detailed diagnosis of this measure to know which operations may potentially fail, why, and at which step.
Skipped events	Indicates the number of provisioning operations where users are skipped.	Number	When a user shows up as “skipped” in the provisioning logs, it is very important to read the extended details in the log message to determine the reason. Below are common reasons and resolutions: A scoping filter has been configured that is filtering the user out based on an attribute value. The user is “not effectively entitled”: If you see this specific error message, it is because there is a problem with the user assignment record stored in Azure AD. To fix this issue, un-assign the user (or group) from the app, and re-assign it again. A required attribute is missing or not populated for a user. An important thing to consider when setting up provisioning is to review and configure the attribute mappings and workflows that define which user (or group) properties flow from Azure AD to the application. This includes setting the “matching property” that be used to uniquely identify and match users/groups between the two systems. Use the detailed diagnosis of this measure to know which provisioning events skipped users and why.
System initiators	Indicates the number of provisioning operations initiated by systems.	Number
Application initiators	Indicates the number of provisioning operations initiated by applications.	Number
User initiators	Indicates the number of provisioning operations initiated by users.	Number
Other initiators	Indicates the number of provisioning operations initiated by factors other than systems, applications, or users.	Number
Total provisioning actions	Indicates the total number of actions performed by provisioning operations.	Number
Create actions	Indicates the number of provisioning operations that attempted to create a user.	Number	Use the detailed diagnosis of this measure to know which operations attempted to create users, when they were performed, which users were to be created, the time it took for each such action to be completed, errors (if any), reason for the errors, and the step at which the errors/failures (if any) occurred. This will point administrators to user creation attemps that resulted in errors. Measures to troubleshoot these errors can also be initiated by insights offered by the detailed statistics. Additionally, these detailed metrics will also if a
Update actions	Indicates the number of provisioning operations that attempted to update a user attribute.	Number	Use the detailed diagnosis of this measure to know which operations attempted to modiofy/update a user account, when such operations were performed, the status of the each such operation, errors (if any), reason for the errors, and the step at which the errors/failures (if any) occurred.
Delete actions	Indicates the number of provisioning operations that attempted to delete a user.	Number	Use the detailed diagnosis of this measure to know which operations attempted to delete a user account, when such operations were performed, the status of the each such operation, errors (if any), reason for the errors, and the step at which the errors/failures (if any) occurred.
Disable actions	Indicates the number of provisioning operations that attempted to disable a user.	Number	Use the detailed diagnosis of this measure to know which operations attempted to disable a user, when such operations were performed, the status of the each such operation, errors (if any), reason for the errors, and the step at which the errors/failures (if any) occurred.
Staged delete actions	Indicates the number of user operations that attempted to disable or delete users but were not permitted to do so.	Number	The Azure AD provisioning service includes a feature to help avoid accidental deletions. This feature ensures that users aren't disabled or deleted in an application unexpectedly. The feature lets you specify a deletion threshold, above which an admin needs to explicitly choose to allow the deletions to be processed. Use the detailed diagnosis of this measure to know which operations violated the deletion threshold. By quickly reviewing these operations, you can decide if you want to allow the deletion or not.
Other actions	Indicates the number of provisioning operations where actions other than user creation, updation, deletion, disabling, were attempted.	Number	Use the detailed diagnosis of this measure to know which operations attempted to other actions, when such operations were performed, the status of the each such operation, errors (if any), reason for the errors, and the step at which the errors/failures (if any) occurred.
Total provisioning steps	Indicates the total number of provisioning steps that were performed across provisioning operations.	Number
Import steps	Indicates the number of provisioning operations that are at the Import step.	Number	This is the step at which the provisioning service attempts to retrieve/import a user from the source system. Use the detailed diagnosis of this measure to know which provisioning operations are at the import step and the status of that step in each such operation. If the Failed events measure reports a non-zero value, then you can use these detailed analytics to figure out if any provisioning events failed at the Import step.
Scoping steps	Indicates the number of provisioning operations where scoping is in progress.	Number	Scoping determines which users are to be provisioned from Azure AD to a SaaS application or from HCM applications (like WorkDay, SuccessFactors etc.) to Azure AD. Use the detailed diagnosis of this measure to know which provisioning operations are at the scoping step and the status of that step in each such operation. If the Failed events measure reports a non-zero value, then you can use these detailed analytics to figure out if any provisioning events failed at the Scoping step.
Matching steps	Indicates the number of provisioning operations that are in the process of matching attributes.	Number	The Azure AD provisioning service can be deployed in both "green field" scenarios (where users do not exist in the target system) and "brownfield" scenarios (where users already exist in the target system). To support both scenarios, the provisioning service uses the concept of matching attributes. Matching attributes allow you to determine how to uniquely identify a user in the source and match the user in the target. Use the detailed diagnosis of this measure to know which provisioning operations are in the process of matching attributes and the status of that step in each such operation. If the Failed events measure reports a non-zero value, then you can use these detailed analytics to figure out if any provisioning events failed at the Matching step.
Processing steps	Indicates the number of provisioning operations that are at the Processing stage.	Number	Use the detailed diagnosis of this measure to know which provisioning operations are in the Processing stage and the status of that step in each such operation. If the Failed events measure reports a non-zero value, then you can use these detailed analytics to figure out if any provisioning events failed at the Processing step.
Resolution steps	Indicates the number of provisioning operations that are at the Resolution stage.	Number	If a matching user isn't found in the target system, it's created using the attributes returned from the source system. After the user account is created, the provisioning service detects and caches the target system's ID for the new user. This ID is used to run all future operations on that user. If a matching user is found, it's updated using the attributes provided by the source system. After the user account is matched, the provisioning service detects and caches the target system's ID for the new user. This ID is used to run all future operations on that user. Use the detailed diagnosis of this measure to know which provisioning operations are in the Resolution stage and the status of that step in each such operation. If the Failed events measure reports a non-zero value, then you can use these detailed analytics to figure out if any provisioning events failed at the Resolution step.
Export steps	Indicates the number of provisioning operations that are at the Export stage.	Number	At this stage, changes are exported to the target system. Use the detailed diagnosis of this measure to know which provisioning operations are in the Export stage and the status of that step in each such operation. If the Failed events measure reports a non-zero value, then you can use these detailed analytics to figure out if any provisioning events failed at the Export step.
Unknown steps	Indicates the number of provisioning operations that are at the Unknown stage.	Number	If a provisioning operation is not in any of the stages discussed above (i.e., import, scoping, matching, processing, resolution, or export), then such an operation is said to be in the Unknown stage. Use the detailed diagnosis of this measure to know which provisioning operations are in the Unknown stage and the status of that step in each such operation. If the Failed events measure reports a non-zero value, then you can use these detailed analytics to figure out if any provisioning events failed at the Unknown step.