Microsoft's Index Server

Adding a search engine

Having great content on your Web site is of no value if users can't find it.I've always wanted to have a great search engine on the Windows NT MagazineWeb site, but providing this feature is easier said than done. Of course, lessthan a year ago, we were hard pressed to find a retail search engine product.Some Web servers had built-in search engines, but they didn't offer much. Today,we have our choice of several Web-based search engines.

The trick to successfully implementing the right search engine is to findone that runs on your Web server. I tried the beta release ofMicrosoft's Index Server (code-named Tripoli) in May 1996. At the time, I passedon using Index Server for our Web site because it required Windows NT 4.0, whichwas still in beta. I didn't want to go that route on our production machine. Ialso had problems getting Index Server to stop indexing and serving everydocument off the Web machine. This behavior explains why Microsoft was promotingit as an intranet search engine. I'm not as concerned about peoplebehind our firewall accessing the company's information, but I'm just not readyto share everything with outside readers and visitors. I also looked at Excite'ssearch engine (Excite for Web Servers--EWS), but it occasionally locked for noapparent reason and brought down the entire Web machine.

Index Server Revisited
I've always had good luck with Microsoft's Internet Information Server's(IIS's) speed and stability. Because Index Server runs on IIS, I decided to lookat Microsoft's search engine again and I downloaded version 1.1 from Microsoft'sWeb site.

Installing Index Server is a breeze. Just a point-and-click here and name adirectory there, and you're finished. I think downloading the program tooklonger than the installation.

The installer asks you for three pieces of information: Where do you wantto store your indexes? Where do you want to store your scripts? Where do youwant to store your sample files? The installer then goes off and startsindexing. When you run your first query, you realize just how many documenttypes this server indexes. The types of documents you can query include HTML andplain text and Microsoft's Word, Excel, and PowerPoint documents. However, thisability is a double-edged sword. Index Server makes all your documents availablefor searching, including documents other than the HTML files in your Webdirectories.

Because I create most of our Web site dynamically with Cold Fusion (as youknow from my December 1996 WebDev, "The FAQs of Web Forums"), most ofmy pages are templates. So when Index Server pulls up a page in the querysummary, the page looks like a text document full of garbage. This situationmeans I have to be careful about what directories I index.

Index Server 1.1 Features
Index Server 1.1 has several nice features. The query results page, as yousee in Screen 1, shows you the number of matches Index Server finds based onyour query and links each match to the original document. The results page alsoprovides an abstract of the document to let you decide whether the document iseven close to what you wanted. At the end of each query, you can view the filesize for each document and the date the document was last updated.

Index Server 1.1 includes new highlighting features. After you post aquery, you can view the contents of each document that the query returns, andthe search criteria appear highlighted. Index Server displays a page from yourquery with every occurrence of the search criteria highlighted in the documentso you can scan quickly to find each instance. And each highlighted word has abackward and forward symbol next to it so you can click through each occurrenceof your queried word. This output is just text--no images.

One area where Index Server 1.1 beats most other packages is in the numberof ways you can query an index. Index Server lets you search according to filesize, modification time, and author property. The modification option isespecially handy. Suppose you want to see all the documents about SQL we've putonline in the last month. You enter SQL in the query box, select in the lastmonth, and start your search. Index Server returns only the specified typeof documents that fit the given timeframe.

Configuring Index Server
Although installing the software is easy, configuring Index Server can betricky. Index Server can easily share every piece of information on your Webserver to your intranet. I had problems restricting which directories I wantedIndex Server to index and make available for query.

Here's the short version of how to limit what information Index Server canaccess and share. From the Administration page, you can select View/Update indexing of virtual roots to select which directories Index Servercan access. The trick is to have the right virtual directories set up in IIS. Asyou can see in Screen 2, I set up some virtual directories (i.e., roots) tospeed the HTML process by not having to hard code every link to every page.Index Server uses these paths to determine what directories it indexes.

By default, Index Server selects all the directories on your Web server. Iwanted only the online magazine articles to be searchable, so I deselected allthe directories except /issues and clicked the Submit changes button.

After you establish your virtual roots, you go back to the Administrationpage and select Force scan virtual roots. You then tell Index Server todo a full scan so that it can index all the documents in those directories. Thisstep limits Index Server's scope.

A more complex way of limiting Index Server's scope is to set the NTpermissions for every directory. You just remove the IIS user account rightsfrom a directory, and Index Server will skip right over it.

I have NT 4.0 Server and IIS Internet Server Manager running on my laptop.To make the changes, I connected to the LAN with Remote Access Service (RAS).First I set the permissions on the search administration directory so that Icould access it via the Web. Next I fired up the IIS Internet Server Manager andcreated all the virtual directories I needed. Then I accessed the Index ServerAdministrator page with Internet Explorer 3.01 (funny how I can't get NetscapeNavigator to work when I do this) and started configuring Index Server.

Now that Index Server 1.1 is set up the way I want, I really like it. Beaware that Index Server's documentation is weak. (At least the guys on the IndexServer team are quick to fess up to the quality of the documentation.) Forexample, when I went to find out about changing the query options, the Help filesaid that I needed to change the .idq file. Unfortunately, Index Server hasabout a dozen of these files, and each one complains rather quickly when youchange it.

Index Server makes managing new directories and updating existing oneseasy. Between Index Server's ease of administration and all that I saw IIS 3.0do at the Microsoft Site Builder conference last October, I've decided to switchfrom our current setup back to IIS.