Optimize Index Server for Faster Response

[Editor's Note: Email your IIS and Site Server solutions (400 words or less) to R2R at [email protected]. We'll edit submissions for style, grammar, and length. If we print your contribution, you'll get $100.]

The users of my Web site were complaining that response to their queries was very slow in the afternoon and evening, but queries seemed to run quickly early in the morning—especially Monday morning. In fact, the slowdown was so noticeable that many users simply stopped using the service late in the day. The situation puzzled me, and I worked to track it down.

I noticed that disk activity later in the day was much higher than it was in the morning—a fact that correlated perfectly with the increase in query response times. I suspected a problem with my Index Server, so I started optimizing it to improve performance. (Although Index Server is a separate Windows NT service, it typically runs in a supporting role to IIS.) Index Server creates word lists and then merges them into shadow indexes. This service stores the word lists in memory but writes the shadow indexes to disk. So the more indexes you have—word lists, shadow lists, master index—the more content that Index Server has to search and the slower any query against Index Server would be.

From the level of disk activity I was getting, I concluded that Index Server was reading the shadow index in processing the queries. Because the shadow index is a disk-based index, reading it for each query could contribute to slower performance. I started performing a master merge to merge the shadow indexes into the master index in the middle of the workday during the company's lunchtime when querying levels dropped off. Users noticed a significant improvement in query response time during the busy hours after I implemented this noontime merge.

—Lee Fields
[email protected]