Using Site Server Express Utilities

Make Web site reporting quick and easy

The Microsoft Windows NT 4.0 Option Pack includes Microsoft Site Server Express 2.0, a subset of Site Server. Site Server Express includes reporting and analysis tools for Internet Information Server (IIS) 4.0 in addition to some usage analysis tools of its own. The Site Server Express tools make analyzing your Web site's traffic a snap. At my company, we used these tools last year when our Web site traffic increased from a few thousand hits a week to more than 100,000 hits a day. We used Site Server Express to generate daily reports so that we could see exactly what was happening with our Web traffic. We could tell, among other things, where referrals were coming from and how many hits we were receiving. This reporting made it easy for us to make intelligent decisions about our site and our marketing efforts.

In this article, I'll take you step by step through using Usage Import and Report Writer—two Site Server Express utilities—to generate reports about your Web site. I'll describe how these tools take information from the IIS log files, and I'll show you how you can configure the tools to automate reporting. Table 1, page 138, describes what Report Writer, Usage Import, and two other Site Server Express utilities can do.

Creating Log Files
The first step in using Site Server Express to create reports is configuring IIS to create log files for your Web server. You configure the logging process from the IIS Microsoft Management Console (MMC) snap-in, and your settings affect the default Web site and all sites under the default site. (To create the log files I set up as examples for this article, I used the WebCat utility from the Microsoft Internet Information Server 4.0 Resource Kit. This tool lets you simulate Web server loads on your IIS server to get a good idea of how the server will perform under various loads.)

Screen 1, page 138, shows the Web Site tab on the IIS Default Web Site Properties page. The default logging options are at the bottom of the form. Check the Enable Logging check box, and IIS will automatically generate logs. You can change the logs' format to suit your needs. Screen 1 shows logging set to the W3C Extended Log File Format. This format can capture a tremendous amount of information about the activities on your server but can also consume a large amount of storage space.

To modify log settings, click Properties to display the Extended Logging Properties form. Then, use the options on the form's General Properties tab to control file-creation periods. If you choose either daily or weekly creation periods, be aware that different log formats determine time differently. For instance, the NCSA Common Log File Format and the ODBC Log Format determine midnight according to the IIS server's local time. However, the Microsoft IIS Log Format and the W3C Extended Log File Format base their determination of midnight on Greenwich Mean Time (GMT).

If you enable the W3C Extended Log File Format, the Extended Logging Properties form includes the Extended Properties tab. This tab gives you control over what information the log records. For instance, you can collect user information, a record of the number of bytes received and sent, and a list of the pages from which users are reaching your site.

Importing Log Files with Usage Import
After you set up your server logging, you can import content and run reports. You use Usage Import to import the log files into the msusage.mdb database. However, before you import log files, you must create a new Log Data Source to define the log source and the type of log data the source contains. Usage Import prompts you to set up the Log Data Source the first time you use this utility. You can also set up the Log Data Source by right-clicking the Log Data Source icon, then clicking New. A wizard takes you through the steps of configuring the server settings.

To import a log, first open the Log File Manager from the File menu or the Usage Import toolbar. Enter the path to the log file in the Log location field. After entering the log file path, you import the log by clicking the green Start import icon on the toolbar. The Import Usage Analyzer will import the data, displaying a summary dialog box with the amount of time the operation took to complete. The Usage Import Statistics dialog box will also appear, providing a glimpse of the contents of the log file. You can scroll through the statistics window to obtain an overview of the log file and to gauge the success of the import. The statistics window will note any problems that occur with records in the log file.

After the import completes, the Log File Manager window, which Screen 2 shows, displays the history of the logs you imported. You can import additional log files by entering a log file's name in the Log location field, then clicking the Start import icon on the toolbar. Each time you import a log file, the file joins the database and the list of log entries in the Log File Manager window.

You can automate the importation of log files with the Scheduler, a graphical interface you access from Usage Import and Report Writer that configures the NT Scheduler. If the NT Scheduler service isn't initialized, Usage Import will start it after you update the schedule. I'll describe this process shortly.

Developing Reports with Report Writer
Now that you've loaded your logs into the database, you can use the Site Server Express Report Writer utility to create reports. Report Writer lets you choose a report from a set of standard report formats or create a custom report. Using either technique lets you quickly design a report, review your creation, and save the report design for later use.

You'll find standard Report Writer formats in the Report Catalog. You create a new report by opening the catalog, choosing a report format, and setting its properties. The property settings let you assign a filter to the report, limiting the report to items you're interested in. These settings make using a report easy—you define the report, set the filter, then save the report. At any time, you can reuse a report you've saved. Also, you can schedule reports in the same way you schedule import tasks. You can use Report Writer as soon as the database contains entries from your log files.

To create a report, start Report Writer from the Site Server Express 2.0 menu from the Option Pack. The Report Writer dialog box appears and lets you either create a new report or open an existing one. Open the Report Catalog by selecting From the Report Write Catalog, then clicking OK to start the Report Writer wizard. This wizard will walk you through the steps of creating your report.

The wizard's first step requires you to choose a particular report format from either detail or summary reports. After you select a report format and click Next, you can set the report's selection options. This step lets you choose what data to review. You can select all the data (the default setting) or restrict the output to a particular date range. After you complete these settings, click Next. The resulting page lets you create a filter to restrict the output of your report. Restricting output is handy when you want to report on only certain types of data and don't want to see anything not related to the data you choose. For now, though, leave the filter blank and click Finish.

Your report will open in a separate window in Report Writer, as Screen 3 shows. You use this window to edit the folders and items (or nodes) the report contains, to expand various elements of the report to review its contents, and to delete nodes from the report. You can also run the report by clicking the Create Report Document icon on the toolbar. You save the report definition by clicking the toolbar's Save icon or pressing Ctrl+S.

When you create a report document, Report Writer processes the report definition against the log file. Then Report Writer creates a set of output files that constitute the report. You can choose one of three output formats: HTML, Microsoft Word, or Microsoft Excel. You can also select which directory will contain the output files. Creating a directory for the files is a good idea because doing so lets you organize the files in a manageable structure. The more reports you create, the more files you must manage; you'll find that creating this structure ahead of time is helpful.

If you choose HTML as your output format, the report output will be a series of HTML pages with supporting graphic files. You can put these files on the Web or on an intranet, or you can view them locally, then delete them.

If you select Word output, Site Server Express first opens Word, then dynamically creates the report. The report will contain text and graphics, just as the HTML version does, but in one Word document. This output is handy when you need to create a report and send it to someone.

If you select Excel output, Site Server Express opens Excel, then generates the report. The report will be in a workbook, with separate sheets for each topic. Separating the report data into sheets makes using the data easy—you can use each sheet's tab to view the various report sections. However, Site Server Express doesn't use Excel functions and graphic features to create the report elements. Instead, the graphics are bitmaps inserted into a sheet, and the various numeric elements are precalculated numbers inserted into a cell. If you want to make changes to the elements, your changes won't update the summary fields (e.g., total, average) and graphic elements. To make the report updateable, you need to recreate the summary fields and graphic elements with linked Excel features.

Automating Import and Reporting Processes
You can automate log importing and report generation by using Site Server Express' command-line features. Usage Import and Report Writer take command-line parameters that let you automate each utility's actions. When you automate the utilities, you can execute them with the NT Scheduler or by creating an automation application. The two command-line executable files are import.exe and analysis.exe. Both files are in the C:Program FilesUA Express directory. The log files for IIS are stored in the C:winntsystem32LogFiles W3SVC1directory by default.

You can manually create batch files to execute log importing or report generation, or you can use the Scheduler interface from Usage Import or Report Writer. The Scheduler interface lets you select the actions you want to use and enter parameters for elements such as dates. The Scheduler interface is the easiest and best way to create batch files, because it creates the files with the correct syntax. The import and analysis executable files are picky about the command-line syntax; creating these files correctly with the Scheduler interface will make your job easier. When you close the Scheduler, it will create the batch file. Then, you can either run the command file or modify it to create a custom file.

The Scheduler works with two variables. A Job is an entity you can schedule, resulting in a batch file for each job. A Task is an action that occurs within a job. For instance, suppose you create a job named ImportAndExecute. This job might have two tasks: ImportLogs and ExecuteReports. A separating header separates each task in a job within the batch file.

To use the Scheduler, open either Report Writer or Usage Import, then click the Scheduler icon on the toolbar. The Scheduler window will open with only the All Jobs folder showing. You can either double-click this folder or right-click the folder and select New Job from the shortcut menu to open the Job Properties dialog box. With this dialog box, you can set options for executing a particular job on a schedule. Also, you can enter a path for the job's log. When you enter the path, you can name the log file and schedule files with consistent names. You can change the default name for a job by clicking it twice slowly in the Scheduler window, then entering the new name.

After you create a job, you can add a task to it. To do so, right-click the job name in the Scheduler window, then select New Task from the shortcut menu to open the Task Properties dialog box, which Screen 4 shows. Select the Task type, which in Screen 4's example is an import operation. If the operation uses a log file, select Log data source from the Log data source list, then select the log file. You can browse in the log file directory and use date-replacement macros to substitute for dates in the filename. For instance, I used $1 to substitute the date for part of the filename. The resulting filename looks like C:winnt system32LogFilesW3SVC1ex$1.log. If the import task runs on November 10, 1998, the filename translates to ex981110.log. To use a replacement macro, select a macro from the list in the log file directory, then drag the macro to the proper location in the filename.

The options in the Task Properties dialog box change depending on the type of action you select. Screen 5 shows the dialog box with the settings configured to create a report. The Definition File field shows that you can select a report definition file you've created with Report Writer. The filename in the Document File field includes a replacement data macro to change the filename to reflect the file's creation date. Notice that the filename is for a Word document. The filename's extension determines the file's type. You can generate HTML output by changing the filename extension from .doc to .htm.

One last note about scheduling files: In Site Server Express 2.0, the Scheduler doesn't order the tasks correctly in the batch file. You can drag tasks in the Scheduler to order the tasks. However, the Scheduler seems to ignore drag-and-drop operations and writes the task order as it sees fit. I tried dragging tasks several times and received varying task ordering results. To reliably change task order, edit the batch file.

As I mentioned earlier, the Scheduler uses the NT Scheduler to manage jobs. When you close the Scheduler interface, you'll receive a warning if the NT Scheduler service isn't running. If you click OK when you receive the warning prompt, you can start the NT Scheduler service.

Many Tools in One
Site Server Express offers a handy set of utilities. You can upgrade from Site Server Express to the full Site Server product to obtain advanced features. Site Server features include support for additional log-file formats, support for SQL Server databases, extensive searching capability, and data aggregation across multiple Web servers and sites. In addition, Site Server provides more than 25 predefined report definitions, lets you create custom usage analysis reports, and can generate detailed advertising reports.

Site Server Express also contains many third-party tools that provide reporting capability and management features for IIS applications. Whether you choose Site Server Express or Site Server, don't forget to keep tabs on new Site Server products and enhancements.