OSS Search Indexer Plugin Test

Using the Search feature of SharePoint 2010, users can easily find the information they need in SharePoint Foundation Sites. One of the key components of the Search architecture is the Indexer component. Also referred to as the Crawl Component or the Crawler, the Indexer is solely responsible for building indexes. The indexers enumerate the source content and pass text information to the relevant index partition on the query server. The indexer also indexes any metadata to the search property database and updates the crawl status in the crawl database.

Using the OSS Search Indexer Plugin test, you can monitor the Indexer component and its activities, swoop down on errors/latencies/CPU contentions in index creation and execution, and understand how these issues impact query processing.

Target of the test : A Microsoft SharePoint Server

Agent deploying the test : An internal agent

Outputs of the test : One set of results for the SharePoint server being monitored

Configurable parameters for the test
Parameters	Description
Test period	This indicates how often should the test be executed.
Host	The host for which the test is to be configured.
Port	The port at which the host server listens.

Measurements made by the test
Measurement	Description	Measurement Unit	Interpretation
Average CPU time create	Indicates the average CPU time that is currently spent to create a query restriction.	Seconds	Compare the values of these measures to know which specific indexing task is hogging CPU - creation of a query restriction? creation of an unsorted cursor? retrieval of a query result set? first stage of ranking? second stage of ranking? query resolution? or any other task?
Average CPU time create unsorted cursor	Indicates the average CPU time that is currently taken to create an initial unsorted cursor.	Seconds
Average CPU time getrows	Indicates the average CPU time that is currently spent to retrieve the result set for a query.	Seconds
Average CPU time other	Indicates the average CPU time that is currently spent in a query apart from Create, Resolve, and GetRows.	Seconds
Average CPU time rank stage 1 with precalc	Indicates the average CPU time that is currently taken for the first stage of ranking with precalculation.	Seconds
Average CPU time rank stage 1 without precalc	Indicates the average CPU time that is currently taken for the first stage of ranking without precalculation	Seconds
Average CPU time rank stage 2	Indicates the average CPU time that is currently taken for the second stage of ranking.	Seconds
Average CPU time resolve	Indicates the the average CPU time that is currently spent to resolve a query.	Seconds
Average execution time create	Indicates the average execution time that is currently spent to create a query restriction.	Seconds	Compare the values of these measures to know where index execution was delayed - creation of a query restriction? creation of an unsorted cursor? retrieval of a query result set? first stage of ranking? second stage of ranking? query resolution? or any other task?
Average execution time getrows	Indicates the average execution time that is currently spent to retrieve the result set for a query.	Seconds
Average execution time other	Indicates the average execution time that is currently spent in a query apart from Create, Resolve and GetRows.	Seconds
Average execution time create unsorted cursor	Indicates the average execution time that is currently taken to create an initial unsorted cursor.	Seconds
Average execution time rank stage 1 without precalc	Indicates the average CPU time that is currently taken for the first stage of ranking without precalculation.	Seconds
Average execution time resolve	Indicates the average execution time that is currently spent to resolve a query.	Seconds
Average execution time rank stage 1 with precalc	Indicates the average execution time that is currently taken for the first stage of ranking with precalculation.	Seconds
Average execution time rank stage 2	Indicates the average execution time that is currently taken for the second stage of ranking.	Seconds
Active connections	Indicates the current number of connections that are active.	Number
Average L0 merge flushes speed	Indicates the average speed (in documents per hour) of L0 merges (flushes) since the catalog was loaded.	Documents/hour
Average L1 merge speed	Indicates the average speed (in documents per hour) of L1 index merges since the catalog was loaded.	Documents/hour
Average L2 merge speed	Indicates the average speed (in documents per hour) of L2 index merges since the catalog was loaded	Documents/hour
Average L3 merge speed	Indicates the average speed (in documents per hour) of L3 index merges since the catalog was loaded.	Documents/hour
Average L4 merge speed	Indicates the average speed (in documents per hour) of L4 index merges since the catalog was loaded.	Documents/hour
Average query latency	Indicates the average latency of queries in the last 1 minute.	Seconds	An unusually high value or a steady rise in the value of this measure implies that query execution is taking too long.
Ongoing L1 merges	Indicates the number of L1 index merges that are currently in progress.	Seconds	If a query is found to be CPU-intensive, you can compare the values of these measures to determine where the query is consuming CPU excessively.
Documents filtered	Indicates the number of documents that have been filtered since the catalog was mounted.	Number
Documents in propagation	Indicates the current number of documents in indexes that are propagated to query components.	Number
Index size	Indicates the size (in MB) of the index.	Bytes
L2 merges	Indicates the current number of L2 index merges since the catalog was loaded,	Number
L3 merges	Indicates the current number of L3 index merges since the catalog was loaded.	Number
L4 merges	Indicates the current number of L4 index merges since the catalog was loaded.	Number
L0 indexes wordlists	Indicates the current number of L0 indexes (Wordlists)..	Number	Wordlists are temporary in-memory indexes
L0 merge flushes	Indicates the current number of L0 index merges (flushes) since the catalog was loaded.	Number
Most recent L1 merge speed	Indicates the speed (in documents per hour) of the last L1 index merge.	Documents/hour
Master merges to date	Indicates the current number of index merges at the master level.	Number
Master merge progress	Indicates the progress of the index merge at the master level.	Number
Master index level	Indicates the current level of the master index.	Number
L0 merge flushes speed	Indicates the average speed (in documents per hour) of L0 merges (flushes) since the catalog was loaded.	Documents/hour
Master merges in progress?	Indicates whether/not any master merges are in progress.	Number	This measure will report the value 1, if a master merge is in progress, and the value 0 if no master merges are in progress.
Ongoing L0 merge flushes	Indicates the number of L0 merges (flushes) since the catalog was loaded.	Number
Ongoing L2 merges	Indicates the the number of L2 merges currently in progress.	Number
Ongoing L3 merges	Indicates the number of L3 merges currently in progress.	Number
Ongoing L4 merges	Indicates the number of L4 merges currently in progress.	Number
Out-of-date document id files	Indicates the number of document ID files with outstanding work items.	Number
Persistent indexes propagated	Indicates the number of persistent indexes propagated.	Number
Persistent indexes	Indicates the current number of persistent indexes.	Number
Persistent L1 indexes	Indicates the current number of persistent L1 indexes.	Number
Persistent L2 indexes	Indicates the current number of persistent L2 indexes.	Number
Persistent L3 indexes	Indicates the current number of persistent L3 indexes.	Number
Persistent L4 indexes	Indicates the current number of persistent L4 indexes.	Number
Queries	Indicates the total number of queries that are currently sent to the server.	Number
Queries succeeded	Indicates the current number of queries that have succeeded.	Number
Queries failed	Indicates the number of queries that have currently failed.	Number	Ideally, the value of this measure should be 0 or very low.
Scope compilations - propagation task receiver	Indicates the the current number of scope compilations that occurred in the propagation task receiver.	Number
Scope compilations - propagation task sender	Indicates the current number of scope compilations that occurred in the propagation task sender.	Number
Shadow merge levels threshold	Indicates the threshold of the shadow merge levels.	Number
Shadow merge levels	Indicates the current level of the shadow index merge.	Number
Tasks - index propagator	Indicates the current number of tasks in the index propagator queue.	Number
Tasks - propagation task sender	Indicates the current number of tasks in the propagation task sender queue.	Number
Tasks - index receiver	Indicates the current number of tasks in the index receiver queue.	Number
Unique keys	Indicates the number of unique keys in indexes.	Number
Up-to-date document id files	Indicates the current number of document ID files with no pending work items.	Number
Work items created	Indicates the current number of work items that are created for updating the indexes.	Number
Work items deleted	Indicates the current number of work items that are deleted because the work items have completed updating the indexes.	Number
Indexes - received successfully	Indicates the number of indexes that have been received by the task propagation receiver since the Gatherer service was started.	Number
Indexes - errors in reception	Indicates the number of errors that occurred in the reception of indexes since the gatherer service was started.	Number	Ideally, the value of this measure should be 0.
Indexes - errors in propagation	Indicates the number of errors that occurred in the propagation of indexes since the gatherer service was started.	Number	Ideally, the value of this measure should be 0.
Indexes - propagated successfully	Indicates the the number of indexes that were propagated since the gatherer service was started.	Number
Propagation rate	Indicates the the current rate at which the documents are propagated to the query components.	Bytes/Milliseconds
Most recent L2 merge speed	Indicates the speed of the most recent L2 merge in documents per hour.	Documents/hour
Most recent L3 merge speed	Indicates the speed of the most recent L3 merge in documents per hour	Documents/hour
Most recent L4 merge speed	Indicates the speed of the most recent L4 merge in documents per hour.	Documents/hour