SharePoint Foundation Search Indexer Test

Using the Search feature of SharePoint 2010, users can easily find the information they need in SharePoint Foundation Sites.

The key components of SharePoint's Search architecture are as follows:

  • Indexer: Also referred to as the Crawl Component or the Crawler, the Indexer is solely responsible for building indexes. The indexers enumerate the source content and pass text information to the relevant index partition on the query server. The indexer also indexes any metadata to the search property database and updates the crawl status in the crawl database.
  • Crawl Database: The Crawl Database tracks what needs to be crawled and what has been crawled.
  • Query Component: Commonly referred to as the Query Server, this component will perform a search against an index created by the indexer. The query component will apply such things as security trimming, best bets, relevancy, removes duplicates, etc.
  • Index partition: Indexes can be split into multiple partitions called index partitions to improve the amount of time it takes to perform a search by the query component. For every query component there will be a single index partition that is queried by the query component.
  • Index Partition Mirror: Mirrors can be created for the index partitions. These mirrors again provide the ability to provide redundancy and better search result performance.
  • Property Database: This database stores metadata and security information items in the index. The property database will be associated with one or more query components and is used as part of the query process. These properties will be populated as part of the crawling process which creates the index.
  • Search Admin Database: The Search Administration Database is mostly responsible for managing information associated to the configuration and topology of the SharePoint Search service. There will only be one instance of this database for each Search Application Service instance.

Figure 1 depicts how these components work together to implement the search feature of SharePoint 2010.

image_6_2CE4C0A7[1]

Figure 1 : How Search works in SharePoint 2010?

When a user enters a search query on a Web Front End (WFE) server, the query server processes the query. While processing, the query server retrieves the information that fulfills the query criteria from the index partition stored on its local file system, and also retrieves metadata information from the search property database. The index partition on the other hand, receives text information from the indexers that enumerate the source content. Once the desired query results are available, the query server packages the results, and delivers the results back to the requesting WFE server.

The success of SharePoint Search feature therefore depends upon how quickly the query server processes the queries it receives, and how effective the index files built by the indexer are.

This test monitors the search queries to the SharePoint server, promptly reports query failures, and thus reveals the overall efficiency of the Search feature offered by Microsoft SharePoint Server.

Target of the test : A Microsoft SharePoint Server

Agent deploying the test : An internal agent

Outputs of the test : One set of results for the SharePoint server being monitored

Configurable parameters for the test
Parameters Description

Test period

This indicates how often should the test be executed.

Host

The host for which the test is to be configured.

Port

The port at which the host server listens.
Measurements made by the test
Measurement Description Measurement Unit Interpretation

Active connections to the indexer plugin

Indicates the number of currently active connections to this indexer plugin.

Number

 

Index size

Indicates the current size of the content index that is being managed by this indexer plugin.

Number

 

Tasks in queue of propagation task sender

Indicates the number of tasks that were in queue of the propogation task sender.

Number

 

Tasks in queue of index receiver

Indicates the number of tasks that were in queue of the index receiver.

Number

 

Tasks in queue of index propagator

Indicates the number of tasks that were in queue of the index propogator.

Number

 

Errors in Index propagation

Indicates the number of errors in index propagation during the last measurement period.

Number

Once the indexer builds the indexes, it propagates/pushes the index files from the index server to the query server. The indexer then waits for the query server to absorb the index, after which it acknowledges that the documents are successfully crawled.

Ideally, no errors should occur in this process - i.e., the value of this measure should be ideally 0. The incidence of one or more errors can adversely impact the user experience with SharePoint's Search mechanism.

Errors in Index reception

Indicates the number of errors in index reception during the last measurement period.

Number

Ideally, no errors should occur in this process - i.e., the value of this measure should be ideally 0.

Indexes received successfully

Indicates the number of indexes that were received successfully by this indexer plugin during the last measurement period.

Number

A high value is desired for this measure. A sudden/gradual decrease in the value of this measure may indicate a performance bottleneck in the Microsoft Server Search Indexer plugin.

Indexes propagated successfully

Indicates the number of indexes that were propagated successfully by this indexer plugin during the last measurement period.

Number

A high value is desired for this measure. A sudden/gradual decrease in the value of this measure may indicate a performance bottleneck in the Microsoft Server Search Indexer plugin.

Documents filtered

Indicates the number of documents that were filtered by this indexer plugin during the last measurement period.

Number

 

Documents in indexes that are being propagated

Indicates the number of documents in indexes which were being propagated by this indexer plugin during the last measurement period.

Number

 

Queries handled

Indicates the number of queries that were handled on the content index during the last measurement period.

Number

 

Successful queries

Indicates the number of queries that were processed successfully during the last measurement period.

Number

A high value is desired for this measure.

Failed Queries

Indicates the number of queries that failed to process during the last measurement period.

Number

Ideally, the value of this measure should be zero.

Avg latency of queries in the last minute

Indicates the average latency with which the queries were processed in the last minute.

Secs

Ideally, when an end user executes a query, results should be returned in less than one second. If this is not the case routinely, then end user experience with the Search feature is bound to suffer. The common reasons for poor query performance and their recommended solutions are as follows:

  • One or more index partitions contain more than 10 million documents: Add an additional index partition, and if possible, an additional index partition mirror. If all query servers already contain an active and a mirrored index partition, add more query servers.
  • One or more query servers are memory bound and/or paging virtual memory on disk: Add additional memory to the query server. Ensure that the query server has enough RAM to store 33% of each index partition (present on the query server) in memory.
  • Query preformance suffers during the first few queries after the server is rebooted or during crawl processing and index propagation: Ensure that the physical disk where the index partition is stored is capable of providing 2,000 IOPS for each index partition.
  • Query latency is high though all query servers are adequately sized: Ensure that the property database server has enough RAM available to store 33% of the property store tables in memory. Make sure that the property database server is not CPU or disk I/O bound. Additional property database servers or property databases can also be added based on need.

Execution time to create a query restriction

Indicates the average execution time to create a query restriction.

Secs

Whenever query latency is very high - i.e., if the Avg latency of queries in the last minute measure reports a very high value - then, you can compare the values of these measures to understand where the query is spending too much time. You can thus identify the bottleneck areas and accordingly decide on the action to be taken to improve query performance. 

Execution time to resolve query

Indicates the average execution time to resolve a query.

Secs

Execution time to get row results of a query

Indicates the average execution time to get row results of a query.

Secs

Execution time spent in other parts of a query

Indicates the average time taken to create a query restriction.

Secs

CPU time to create a query restriction

Indicates the average CPU time that is required to create a query restriction.

Secs

 

 

If a query is found to be CPU-intensive, you can compare the values of these measures to determine where the query is consuming CPU excessively. 

CPU time to resolve a query

Indicates the average CPU time taken to resolve a query.

Secs

CPU time to get row results for a query

Indicates the average CPU time taken to get row results of a query.

Secs

CPU time spent in other parts of a query

Indicates the average CPU time taken to execute other parts of the query.

Secs