Windows Search


Windows Search is a content index and desktop search platform by Microsoft introduced in Windows Vista as a replacement for the previous Indexing Service of Windows 2000, Windows XP, and Windows Server 2003, designed to facilitate local and remote queries for files and non-file items in the Windows Shell and in compatible applications. It was developed after the postponement of WinFS and introduced to Windows several benefits of that platform.
Windows Search creates a local Index of files — audio tracks, documents, folders, programs, photos, and videos — as well as of non-file items such as contacts and messages of Microsoft Outlook and their metadata for which users can perform incremental searches based on details such as contents, dates, names, types, and sizes; the Index consists of prose within files and items and metadata properties. Control Panel and Settings can also be searched.
Windows Search was introduced to enhance search result rapidity, simplify data discovery, and to unify desktop search platforms across Microsoft Windows; it was available as an optional download for Windows 2000 and Windows XP. It has received several updates since its introduction and it is also available in Windows 7, Windows 8, Windows RT, Windows 8.1, Windows 10, and Windows 11.
In January 2025, Microsoft released an enhanced version of Windows Search for their new NPU-based Copilot+ PC devices. This release of Windows Search additionally enables finding local documents, images and settings using user's own words and phrases, instead of just relying on exact keyword matches.

History

Windows Search is the successor of the Indexing Service, a content indexing solution originally developed as an optional download for Windows NT 4.0 and Internet Information Services 3.0, designed to gather resources located on Web servers; it is a remainder of the Object File System of the Cairo operating system project that never fully emerged. Development of Windows Search began in 2004 after the postponement of WinFS, the common data storage platform developed for Windows Vista, as an optional component of its MSN Toolbar Suite for Windows XP and Windows 2000 released as preliminary software on December 13, 2004.
In prerelease builds of Windows Vista it was integrated with the Windows Shell, renamed as the Windows Search Engine, and introduced features that were originally touted as benefits of WinFS: content indexing, incremental searching, property stacking, and query persisting. Windows Search like WinFS enables queries across existing commonalities between items but unlike WinFS it does not provide a relational item database with schema-based enforcement, management, and storage; it provides a way to search for commonalities that already exist across disparate types.
Windows Search additionally deprecates the Indexing Service, as WinFS was originally meant to do. Microsoft ultimately removed the Indexing Service from Windows with the release of Windows 8 in 2012.

Overview

Windows Search builds a local Index of files and non-file items stored on a machine, which allows results to appear more rapidly when users perform searches than when searching for unindexed files or items. Microsoft suggests that a few hours may be necessary for the building process to complete, an assessment dependent on the number and size of files and non-file items to index. Windows Search enables incremental search to present search results immediately when a character is typed in a search box, with subsequent entered characters further narrowing and refining results even before the full name, phrase, or word is entered. No information in the Index is sent to Microsoft.
Windows Search is an extensible and programmable platform with multiple interfaces for operating over new or preexisting applications, data stores, file systems, and types:
InterfacePurpose and use in Windows Search
IFilters Extract, parse, and scan file contents
Property HandlersEmit properties of files and items
Protocol HandlersExpose data in data stores and file systems

  • IFilters — the same interface used by Microsoft Exchange Server, SharePoint, and SQL Server — are used to extract, parse, and store in the Index the contents of various file formats. Default IFilters are available for over 200 common file formats. IFilters can be created for new or proprietary formats.
  • Property Handlers of the Microsoft Property System introduced in Windows Vista expose metadata properties of files and non-file items to the Index for rapid retrieval in response to user queries; property handlers can be created for new and proprietary files and non-file items.
  • Protocol Handlers enable files and non-file items in repositories of proprietary applications or file systems to be indexed by the Indexer and exposed to the Windows Shell. There are inbox protocol handlers for the File System, Internet Explorer, Microsoft Outlook, and Offline Files.

    Architecture

Windows Search is implemented as a Windows Service. The search service implements the Windows Search configuration and query APIs and also controls, as well as all indexing and query components. The most important component of Windows Search is the Indexer, which crawls the file system on initial setup, and then listens for file system notifications to pick up changed files in order to create and maintain the index of data. It achieves this using three processes:
  1. SearchIndexer.exe, which hosts the indexes and the list of URIs that require indexing, as well as exposes the external configuration and query APIs that other applications use to leverage the Windows Search features.
  2. SearchProtocolHost.exe, which hosts the protocol handlers. It runs with the least permission required for the protocol handler. For example, when accessing filesystem, it runs with the credentials of the system account, but on accessing network shares, it runs with the credentials of the user.
  3. SearchFilterHost.exe, which hosts the IFilters and property handlers to extract metadata and textual content. It is a low integrity process, which means that it does not have any permission to change the system settings, so even if it encounters files with malicious content, and by any chance if they manage to take over the process, they will not be able to change any system settings.
The search service consists of several components, including the Gatherer, the Merger, the Backoff Controller, and the Query Processor, among others. The Gatherer retrieves the list of URIs that need to be crawled and invokes proper protocol handler to access the store that hosts the URI, and then the proper property-handler and IFilter to extract the document text. Different indices are created during different runs; it is the job of the Merger to periodically merge the indices. While indexing, the indices are generally maintained in-memory and then flushed to disk after a merge to reduce disk I/O. The metadata is stored in property store, which is a database maintained by the ESE database engine. The text is tokenized and the tokens are stored in a custom database built using Inverted Indices. Apart from the indices and property store, another persistent data structure is maintained: the Gather Queue. The Gather Queue maintains a prioritized queue of URIs that needs indexing. The Backoff Controller mentioned above monitors the available system resources, and controls the rate at which the indexer runs. It has three states:
Image:Windows Search.svg|thumb|right|Windows Search architecture|240px
  1. Running: In this state, the indexer runs without any restrictions. The indexer runs in this state only when there is no contention for resources.
  2. Throttled: In this state, the crawling of URIs and extraction of text and metadata is deliberately throttled, so that the number of operations per minute is kept under tight control. The indexer is in this state when there is contention for resources, for example, when other applications are running. By throttling the operations, it is ensured that the other operations are not starved of resources they might need.
  3. Backed off: In this state, no indexing is done. Only the Gather Queues are kept active so that items do not go unindexed. This state is activated on extreme resource shortage, or if indexing is configured to be disabled when the computer is on battery power, or if the indexer is manually paused by the user. If the Windows Search service is stopped and disabled, the search function of Windows Explorer is still usable but search speed maybe slower.

    Advanced Query Syntax

Windows Search queries are specified in Advanced Query Syntax which supports not only simple text searches but provides advanced property-based query operations as well. [|AQS] defines certain keywords which can be used to refine the search query, such as specifying Boolean operations on searched terms as well as to specify further filters based on file metadata or file type. It can also be used to limit results from specific information stores like regular files, offline files cache, or email stores. File type specific operators are available as well. Windows Desktop Search also supports wildcard prefix matching searches. It also includes several SQL-like operators like GROUP BY. AQS is locale dependent and uses different keywords in international versions of Windows 7.

Programmability

Users can access the Windows Search index programmatically using managed as well as native code. Native code connects to the index catalog by using a Data Source Object retrieved from the Indexing Service OLE DB provider. Managed code use the MSIDXS ADO.NET provider. One can query a catalog on a remote machine by specifying a UNC path. Programmers specify the criteria for searches using SQL-like syntax. The SQL query can either be created by hand, or by using an implementation of the ISearchQueryHelper interface. Windows Search provides implementations of the interface to convert an AQS or NQS queries to their SQL counterpart.
The OLE DB/SQL API implements the functionality for searching and querying across the indices and property stores. It uses a variant of SQL in which to represent the query and returns results as OLE DB Rowsets. Whenever a query executes, the parts of the index it used are temporarily cached so that further searches filtering the result set need not access the disk again, in order to improve performance. Windows Search stores its index in an Extensible Storage Engine file named Windows.edb that exists, by default, in the \ProgramData\Microsoft\Search\Data\Applications\Windows\ folder at the root of the system drive in Windows Vista or in later versions of Windows.
The index store, called SystemIndex, contains all retrievable Windows IPropertyStore values for indexed items.
Within the SystemIndex folder lurk and files.
The names and locations of documents in the system are exposed as a table with the column names System. ItemName and System. ItemURL respectively. A SQL query can directly reference these tables and index catalogues and use the MSIDXS provider to run queries against them. The search index can also be used via OLE DB, using the CollatorDSO provider. However, the OLE DB provider is read-only, supporting only SELECT and GROUP ON SQL statements.
Windows Search also registers a search-ms application protocol, which can be used to represent searches as URIs. The search parameters and filters are encoded in the URI using AQS or its natural language counterpart, NQS. When invokes the URI, Windows Search launches the Search Explorer with the results of the search. In Windows Vista SP1 or later, third-party handlers can also register themselves as the application protocol handler, so that searches can be performed using any search engine which the user has set as default, and not just Windows Search.
The Windows Search service provides the Notifications API component to allow applications to "push" changed items that need indexing to the Windows Search indexer. Applications use the component to supply the URIs of the items that need to be indexed, and the URIs are written to the Gather Queue, where they are read off by the indexer. Microsoft Outlook and Microsoft OneNote use this ability to index the items managed by them and use Windows Search queries to provide the in-application searching features. The internal USN Journal Notifier component of Windows Search also uses the Notifications API, monitoring the Change Journal in an NTFS volume to keep track of files that have changed on the volume. If the file is in a location indexed by Windows Search and does not have the FANCI attribute set, the Windows Search service is notified of its path via the Notification API.
Windows Search Configuration APIs are used to specify the configuration settings, such as the root of the URIs that needs to be monitored, setting the frequency of crawling or viewing status information like number of items indexed or length of the gather queue or the reason for throttling the indexer.
It also exposes APIs to register protocol handlers interface, property handlers or IFilter implementations. IFilter implementations allow only read-only extraction of text and properties, whereas IPropertyStore allows writing properties as well.