Making Sense of SharePoint Search

In a SharePoint environment, users can perform searches from team sites hosted by Windows SharePoint Services and from portal sites hosted by Microsoft Office SharePoint Portal Server 2003. (For definitions of team sites, portal sites, and other types of sites, see the sidebar "Types of SharePoint Sites.") SharePoint Services and Portal Server use the same search engine, but the different ways they use that engine result in an inconsistent search experience for end users.

Learning about the two products' different search architectures can help you decide whether your users would benefit most by using SharePoint Services, Portal Server, or both together. If you haven't yet deployed SharePoint, understanding SharePoint search architecture will help you plan for an eventual deployment.

Major Search Components
Microsoft Search Service (MSSearch) is the generic search service that Microsoft products, including SharePoint Services and Portal Server, use to varying degrees. It supports three query languages—Query Dialect 1, SQL full-text extensions, and Query Dialect 2—which let users perform varied forms of queries. MSSearch creates full-text indexes on content and properties of structured and semistructured data and allows fast linguistic searches on this data.

For an index to be truly useful, however, it must support the content sources that users are interested in (e.g., file shares, Web content, Exchange Server folders, SharePoint-based sites) and the types of data within those content sources, and it must offer rich retrieval capabilities. The applications that use the index need to provide the content to be indexed, format user queries, and return the results of searches to the requesting clients. Consequently, these applications in effect control the overall search experience, and some exercise that control much better than others. Portal Server, for example, provides content to be indexed from many sources, such as Exchange public folders and IBM Lotus Notes databases, whereas SQL Server full-text indexing provides content only from SQL Server tables.

MSSearch relies on several major components to support full-text indexing. Figure 1 shows the following components:

Individual products also contribute to MSSearch's search architecture. For example, Portal Server adds Alerts to the search architecture, and third parties can provide filters and protocol handlers. However, not all products benefit from all of MSSearch's search components. For example, only Portal Server uses Alerts, but any product that uses the MSSearch architecture will benefit from IFilters.

Now that we understand a little about the search architecture, let's see how SharePoint Services and Portal Server differ in the way they use the architecture. We'll also look at how this difference results in different outcomes for end users searching SharePoint sites.

Searching with SharePoint Services
SharePoint Services relies on SQL Server full-text indexing, which in turn interacts with MSSearch. Thus, the search capabilities that SharePoint Services can offer to team sites are limited by what SQL Server full-text indexing can do. One prerequisite for enabling search within a team site is that SharePoint Services must use SQL Server rather than Windows MSDE (WMSDE) as its storage engine. By default, full-text indexing is disabled, but you can enable it by using the SharePoint Central Administration page. No search links will appear on team sites unless you enable full-text indexing.

SQL Server full-text indexing provides a protocol handler for MSSearch that understands only the content of SQL Server databases and tables. Therefore, the results of a search initiated from a team site include content only from SQL Server. When you enable searching in SharePoint Services, you enable it for all team sites because content from different team sites isn't separated into different tables. For example, the Docs table in a content database contains documents stored in all libraries within all team sites that use that content database.

A search from a team site supports stemming and inflectional forms (i.e., different forms of a word) of the supplied query terms. In SharePoint Services, you're limited to using exceptionally basic search terms, and neither phrase nor Boolean searches are supported. Additionally, you can't narrow a search to within a previous result set, and you won't see any indication of the number of hits. Although you can limit a search's scope to the contents of an individual list, the search is performed against the current view of that list and won't return results for items that are filtered out of that view.

SharePoint Services' search capabilities are fairly inflexible and not designed for enterprise use. SharePoint Services indexes document content only, not any properties that might further describe the document's contents. Even if the search engine did index document properties, the query interface doesn't support advanced search, so users wouldn't find that information. Users are limited to doing a full-text search on content only. Furthermore, SharePoint Services' search capabilities are limited in scale, severely restricting their use in large server farms, which could conceivably contain millions of documents across thousands of team sites.

Searching with SharePoint Portal Server
Portal Server offers enterprise-strength search capabilities. Although the underlying indexing architecture hasn't changed much since SharePoint Portal Server 2001, its scalability, performance, manageability, usability, and accessibility have significantly improved.

Out of the box, Portal Server provides protocol handlers for any kind of SharePoint site, file shares, Exchange public folders, Lotus Notes, and any Web site. The support for SharePoint sites means searches can include information about documents and list data from team sites, the site directory, and information about people stored in the Portal Server profile database. Unlike SharePoint Services, Portal Server can expand the scope of a search to a SharePoint Services team site and all its subsites and to an entire Portal Server portal implementation.

Portal Server includes IFilters for Microsoft Office documents and for HTML, TIFF, and text files. The TIFF IFilter enables Portal Server to crawl the textual content of saved fax data based on optical character recognition (OCR) technology. Portal Server uses the MIME IFilter that ships with Windows 2000 to filter messages from Exchange public folders.

Portal Server indexes not only textual content but properties associated with an item. Properties can be associated with source content in different ways and are generally used to further describe the content. For example, Microsoft Office documents have built-in and custom properties, meta tags are properties within HTML files, and columns in lists are properties of SharePoint sites. By indexing properties, you can perform much more refined searches than you can with a full-text search of only the content. For example, indexing properties helps you to search for people who belong to a particular department.

From a query language point of view, Portal Server supports SQL Server full-text extensions, which let you do Boolean searches, stemming, inflection, and advanced searches based on properties. End users can perform simple or advanced searches using the out-of-the-box Search Web Part.

Also, Portal Server provides a Search Web Service for customizing search solutions. For example, Word users can use the Search Web Service to search portal sites directly through the Research task pane, as Figure 2 shows.

Besides more useable indexing and a more efficient search experience, the biggest benefit of Portal Server from the end user's point of view is how search results are displayed. Using the Search Web Page, the user can easily find several dynamic canned views and manipulate the search results set, enabling smart groupings and sorting. Figure 3 shows a search results set grouped by author. Additionally, the user can choose to show less or more information and can set up an alert for the search query that generated the search results set.

Combining SharePoint Search Methods
In a default Portal Server deployment that uses a SQL Server back end, full-text indexing is enabled for team sites. Full-text indexing means that the search UI will appear on any team site that's created using Portal Server. However, the search UI for a team site is different than the search UI at the portal level. At the portal level, the page that loads is search.aspx, and it loads the SearchResults Web part. However, from a team site that's part of the portal site, the SearchResults page is loaded and although it might use the SearchResults Web part, the UI is different. Additionally, team sites created via Portal Server rather than from a standalone SharePoint Services system are automatically "associated" with the portal. You'd think that since the team site was created from the portal site, any search performed from the team site would somehow execute against the portal, but that isn't the case. Even though the team site is created from a portal site, it still uses the SharePoint Services search mechanism rather than the Portal Server mechanism.

To take advantage of the greater search capabilities in Portal Server, the user must execute the query against the portal site, if the team site is associated with the portal site. For an associated team site, a link in the local search results redirects the user to the portal site. Also, a navigation aid placed on the team site lets the user navigate to the portal site. If the team site isn't associated with the portal site or if full-text indexing has been disabled, the user won't be able to get from the team site to the portal site to do a search.

You can customize the team site to allow searching on the portal site. For example, you can design a SharePoint Services site template to include a Search Web Part that automatically searches on the portal. Also, third-party companies have developed Web Parts to make SharePoint Services searching more effective.

Looking Ahead
The ability to search SharePoint sites and get accurate results is important to users. Your business has valuable content stored in SharePoint, and people can't collaborate successfully unless they can find that content. You can help users search SharePoint more effectively and efficiently by carefully planning and giving some thought as to how best to partition your indexes and content sources, as well as how to integrate Portal Server search into other team sites and, perhaps, into external applications.