Project Server Queue Jobs Test

Nearly all critical operations/jobs in the Project Server go through the Queuing System. These include:

Project Save
Project Publish
Timesheet Save
Timesheet Submit
Project Backup/Recovery
Report Data Service operations
Cube Building Service operations
Server Side Scheduling (and node-consistency processing)

If the Project Server is unable to process these jobs quickly, it would result in an overload condition characterized by long-winding job queues and a general server slowdown. In the event of such abnormalities, administrators will have to instantly figure out which type of jobs are contributing to the overload and why – is it because jobs of this type are failing frequently owing to errors? Or is it because the server is not adequately configured to handle these jobs? The Project Server Queue Jobs test helps administrators answer these questions!

This test auto-discovers the type of jobs in queue, and for each job type, reports the count of jobs processed, the processing time of jobs, and the percentage of job failures. This way, the test sheds light on job types that fail often, those that are taking too long to complete, and the probable reasons for the same.

Target of the test : A Microsoft Project Server

Agent deploying the test : An internal agent

Outputs of the test : One set of results for each job type that is to be monitored.

Configurable parameters for the test
Parameters	Description
Test Period	How often should the test be executed.
Host	The host for which this test is to be configured.
Port	The port at which the specified host listens. The default port is Null.

Measurements made by the test
Measurement	Description	Measurement Unit	Interpretation
Average processing time	Indicates the average time taken by the Project server Queue system to process the jobs of this type.	Minutes	The value of this measure would vary based on the type and size of the job that is being processed. Ideally, the value of this measure should be low. A consistent increase in the value of this measure for any job type could indicate a bottleneck when processing jobs of this type. Administrators may then want to investigate further to figure out where the bottleneck lies.
Average wait time	Indicates the average time the jobs of this type had to wait in queue to be processed by the Project server Queue system.	Minutes	Ideally, the value of this measure should be low. A high value indicates that the jobs are spending too much time in queue. Possible causes for this abnormality are as follows: Jobs are blocked in the queue of the Project Server application server. The Project Server is overloaded The Project Server application server may have fewer job processor threads than required to meet the inflow of jobs To address these issues, administrators can do the following: Verify whether jobs are blocked in the queue by reviewing the Windows Event log and trace logs, and troubleshoot accordingly. If the problem persists, determine whether the problem is due to an overloaded Project Server by monitoring the following performance counters on the affected server Try increasing the number of job processor threads
Jobs retried per hour	Indicates the percentage of jobs of this type that were retried over the past hour by the Project server Queue system.	Percent	Ideally, the value of these measure should be zero. A high value for these measures could indicate one/all of the following: Job failures are high; The retry interval setting for jobs that failed due to SQL-related issues (such as SQL deadlocks), is small;
Jobs retried per minute	Indicates the number of jobs of this type that were retried during the last minute by the Project server Queue system.	Number
Jobs failed per hour	Indicates the percentage of jobs of this type that failed over the last hour in the Project server Queue system.	Percent	Ideally, the value of these measures should be zero. A non-zero value indicates that one/more jobs failed; this is a cause for concern and requires investigation. Perform the following steps to determine the reason for these failures and troubleshoot accordingly. View errors in Microsoft Project Web App: Log on to Project Web App as an administrator. In the Quick Launch, click Server Settings. On the Server Settings page, in the Queue section, click Manage Queue. On the Manage Queue Jobs page, find the jobs that have the job state of Failed and blocking or Failed and not blocking correlation. View more information about the failed jobs by clicking the error IDs that appear in the Error column of the failed jobs. View errors in the Windows Event log of the Project Server to determine whether any errors occurred at around the same time as the job failures. View the trace logs on the Project Server to determine whether any errors occurred at approximately the same time as this alert that may have caused the problem.
Jobs failed per minute	Indicates the number of jobs of this type that failed over the last minute in the Project server Queue system.	Number
Jobs processed	Indicates the number of jobs of this type that were processed per minute by the Project Server Queue system.	Number	A high value is desired for this measure. A consistent decrease in the value of this measure clearly indicates a processing bottleneck.