Archiving and Analyzing the NT Security Log

Free and inexpensive tools help you track event-log activities

Windows NT thoroughly captures security events, but the OS doesn't provide many tools to track those events. Given the difficulty and time-consuming nature of manual event-log analysis, you might neglect Security-log analysis activities. But if you ignore these important tasks, you might lose security-event data or miss the signs of an attempted (and possibly successful) attack. In this final article of the NT Security log series, I describe several free or inexpensive tools that can help you archive, merge, and analyze your systems' event logs. (For information about the previous articles, see "Other Articles in This Series," page 82.) I also show how to import your logs into one Microsoft Access database so that you can write custom reports.

Archiving Event Logs
As I explain in "Introducing the NT Security Log," March 2000, you can keep only a limited history of events in the Security log. Regardless of the maximum log size and event-wrapping options that you use, you eventually need to clear the log or overwrite older events. I recommend that you use your job scheduler (e.g., the NT Scheduler service) and a batch-file utility to archive the Security log before each nightly tape backup. If you choose a log size that accommodates the average amount of recorded activity that occurs between backups, you'll have a contiguous Security log archive that dates back as far as your tape rotation permits.

To automate event-log archival, you need a batch-file utility that can import your Security logs into an output file. The following four tools have certain strengths and peculiarities; one of the four will likely suit your environment. (For information about other available tools, see Mark Joseph Edwards, "The Handy Security Toolkit Revisited," October 1999.)

Dumpel. The Microsoft Windows NT Resource Kit's Dumpel is a well-known utility that can dump NT's three event logs (i.e., Security, System, and Application) to an output file. Dumpel lets you specify the log to dump, an output file, and filtering criteria. The utility dumps the log only in text format, delimiting each record with a carriage return or line feed. To copy the local system's Security log to a file called sec_archive.txt, type

dumpel ­f sec_archive.txt ­l security

Unfortunately, the utility doesn't let you dump an event log in the log's native .evt file format, which NT Event Viewer uses. Dumpel's inability to save .evt output files is this tool's primary flaw. However, you can use the utility's c parameter or t parameter to specify comma-delimited or tab-delimited columns, respectively. So ostensibly, you can easily import the files into a database such as an Access database, from which you can view and write reports. However, Dumpel's format for each event record's free-form description field doesn't match the Access parsing algorithm and causes conversion problems.

The description field is a series of text strings; each event ID has a different set of strings. Some strings are dynamic (i.e., different for each occurrence of the event ID). Others are static (i.e., identical for each occurrence). Figure 1 shows an example description of event ID 528. The only information that NT stores in the event log is the dynamic string—for example, Administrator, DOMAIN, (0x0,0x3779C), 7, User32, Negotiate, and TECRA. NT stores the static string (i.e., Successful Logon:, User Name:, Domain:, Logon ID:, Logon Type:, Logon Process:, Authentication Package:, and Workstation Name:) in a separate standard message file. The Event Viewer and other programs that dump event logs in text file format must merge the dynamic and static strings. This requirement presents no problem for events that native NT components generate because NT's standard message files are present on every NT system. However, when you use third-party drivers and applications to generate an event from a particular system, that event might display correctly only on that system because the application's static message is unavailable on other systems. In this situation, the event description notes that the message text is unavailable and displays only the event's dynamic data string. (For example, you might see only the dynamic data string when you view a Microsoft SQL Server event from a workstation that doesn't have SQL Enterprise Manager installed.)

A key aspect of event-log archiving is the need to collect logs from multiple systems. Many systems administrators prefer to keep all the systems' event logs in a central directory. Dumpel lets you use the ­s parameter followed by a computer's name to retrieve and dump a remote system's event logs.

EventSave. Frank Heyne Software's EventSave (which you can download free at http://www.heysoft.de/) copies all three event logs in .evt file format and thus addresses Dumpel's primary flaw. EventSave collects the event logs from each of your crucial systems and maintains the logs in a central archive. The program checks the recorded time of each event in each log, then sorts the events into one file per log per computer per month. The program uses the format year_month_computername_eventlogname for these files. For example, on a system with event logs that contain events only from April 2000, EventSave will produce three files: 2000_04_oraclesvr_security.evt, 2000_04_oraclesvr_system.evt, and 2000_04_oraclesvr_application.evt. When a destination file already exists (e.g., because you previously ran EventSave that month), the tool appends new events to the existing file so that regardless of how often you run EventSave, you end up with only one file per month.

You can use the tool's /A switch to automatically save the event logs of each system in a domain, or you can use the /C switch to specify one computer. (If you want a subset of the domain's systems, you must write a batch file that executes EventSave for each computer.) For example, to save server2's event logs to the eventlogs shared directory on server1, type

eventsave \server1eventlogs /Cserver2

If you don't specify an archive path, EventSave puts the archive files in the directory that is active; if you don't add a computer name after the /C switch, the program saves the archive files in the local system's event logs.

One warning about EventSave: The tool unconditionally clears the logs after saving them, so you can't keep a constant amount of activity in each system's logs. If this restriction is a problem for you, consider Frank Heyne Software's EventCopy, which functions like EventSave but doesn't clear the logs. To archive the logs and still keep a month of activity on each system, you can set your event logs to overwrite events older than 30 days, then run EventCopy daily to add new events to the central archive directory. Frank Heyne Software also offers EventSave+, which lets you specify an event log to save for each system. The vendor negotiates pricing for these two utilities on a case-by-case basis.

EventSave and its siblings use Win32 event-log-dumping APIs that save event logs only to the local computer's hard disk. To support remote saves across a network, EventSave instructs the remote computer to dump the log to its local %systemroot%system32config directory, then the tool uses file sharing to copy that information to the specified output directory. The remote systems must run the NT Server service, and if you've disabled administrative shares, you need to share the boot drive's root and use eventsave's /S switch to specify the share to EventSave. The user account that you use to run EventSave also requires change access to %systemroot%system32config, read access to the HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesEventLog Registry key, and rights to Manage Auditing and Security Log on each system for which the utility accesses event logs.

NTOLog. NT OBJECTives' NTOLog (which you can download as part of NTO Tools at http://www.ntobjectives.com/) also makes an .evt backup of your local and remote systems' event logs. The free product uses a date and time filename format (i.e., DD-MM-YY-HH-MM-SS) and an extension (i.e., .secbak, .sysbak, or .appbak) that depends on the type of saved event log. NTOLog lets you use the /sec, /sys, or /app switches to save only the Security, System, or Application log, respectively. For example, if you type

ntolog \oraclesvr /sec

at 10:02:32 on April 3, 2000, NTOLog will create a file called 03-04-00-10-02-32.secbak in OracleSvr's C:winntsystem32config directory. You can also use the tool's /c switch to clear a log after saving it.

NTOLog doesn't let you choose where to put the output files; the tool puts the files in the C:winntsystem32config directory of the archived logs' system. This restriction can be a problem if that path is invalid on your system. NT OBJECTives plans to remove this hard-coded path, but until then you can use a binary file editor to modify the NTOLog executable. And although NTOLog can help archive and back up your event logs, the product doesn't offer a way to automatically distinguish and collect output files from remote systems. If you want to maintain a central event-log archive, you need to append a computer name to each output filename, then copy the output files to a central directory. I recommend that you use NTOLog if you want to dump event logs only in .evt format for inclusion in your backups; use EventSave, which produces a clean set of appropriately named monthly files for each system, if you want to maintain a central online archive.

ELDump. You might consider Jesper Lauritsen's ELDump (which you can download free at http://www.ibt.ku.dk/
jesper/eldump/) if you need to dump your event logs in text format but require more functionality than Dumpel offers. The product is especially useful for importing the logs into an Access or SQL Server database. ELDump dumps a system's current event logs or lets you read .evt files that you previously dumped using other utilities. The tool provides excellent flexibility for specifying which events to output (e.g., events for a particular time period or specific event ID). ELDump also provides control over the output data format, including the output file's date and time format.

Another notable feature is ELDump's method of handling each event record's free-form description field. To prevent the display problem that I described earlier, ELDump uses the remote system's message files when dumping a remote event log.

Merging Event Logs
In "Introducing the NT Security Log," I explain how NT's fragmented logging nature isolates each system's Security log and provides no way to assess security events for your entire network (e.g., scan for all failed logon attempts throughout the network). If you're responsible for multiple systems, you need a way to merge the saved event logs from all systems into one file from which you can track events for the network. If you want one .evt file that contains all your systems' Security logs, try Frank Heyne Software's Merge Eventlog Records (MER), which is available with the registered version of Report Event for Windows NT/2000. This utility imports the .evt files that EventSave creates and uses them to produce one output file. You can then use Event Viewer to view your entire network's security activity. You can specify a wildcard filename (which tells MER which log files to read) and a destination filename. You can use the /E switch followed by comma-delimited event IDs to specify the event IDs that you want the program to merge. For example, to scan all your computers' Security logs that EventSave archived in C:events for May 2000 and write logon failure event ID 529 and event ID 539 to a new file (e.g., 2000_05_failedlogon_security.evt) in the default (i.e., active) directory, type

mer C:events2000_05*security.evt 2000_05_failedlogon_security.evt /E529,539

Or to create one file (e.g., securitymerged.evt) that comprises all the Security logs from EventSave's archived directory, type

mer C:events*security.evt securitymerged.evt

Analyzing Event Logs
Now that you know how to collect and merge many event logs into one file, how do you efficiently analyze this file to find suspicious activity patterns? Several available reporting utilities can analyze output files for specific types of activity.

R528. R528 helps you monitor users' logon sessions to determine who accessed a system during an incident, determine normal usage trends, and recognize anomalous logons that might indicate a security breach or unauthorized account use. The utility searches for occurrences of event ID 528 (successful logon) and corresponding event ID 538 (user logoff) in one or more .evt files. (For more information about logon events, see "Monitoring Privileges and Administrators in the NT Security Log," June 2000.) R528 lets you choose from several report sections, which Figure 2 shows.

The first section details each logon session, showing the user who logged on, which computer the user logged on to, the logon type, the logon date and time, and the logoff date and time. One problem with NT's auditing is that NT doesn't always record event ID 538; consequently, R528 will probably find more records with event ID 528 than event ID 538.

The utility's second report section lists the unmatched logon events. Note that R528 reports show the system that a user logged on to, not the system the user logged on from. (For example, when a user logs on to a local workstation with a domain account, NT records the logon in the local workstation's log. When the same user accesses a shared directory on another server, NT records that logon in the other server's log.)

You can use R528 to search multiple .evt files simultaneously and produce one comma-, tab-, or semicolon-delimited report. R528 has a host of options to control the reported information and level of detail, so you might need to experiment to find the right reporting combination for your needs.

R529. R529 helps you monitor event ID 529 (failed logon). The utility's report (which Figure 3, page 85, shows) lists failed logon attempts, detailing the user account, the system the user attempted to log on to, and whether the logon originated locally or over the network. R529 also tries to distinguish between innocent mistakes and attacks by looking for two or more failed logons that involve the same account and computer and that occur within 5 minutes of one another. When R529 finds a successful logon that immediately follows a failed logon, the utility reports the pattern as a mistake. When a successful logon follows a string of failed logons, R529 reports a successful attack. For all other situations, the utility reports a failed attack. The utility's simplistic pattern recognition can't dependably recognize all attacks, but the tool's mistake recognition helps you weed out failed logons that were likely the result of user error. Two optional summary sections help you identify the computers and user accounts that intruders attack most frequently.

NTLast 1.6. NT OBJECTives' NTLast 1.6 is another free tool that you can use to analyze logon activity. (The company also offers NTLast 2.85 for purchase; for a review of NTLast, see Mark Joseph Edwards, "NTLast v2.6," September 1999.) Similar to the UNIX last command, NTLast offers several reporting options to help you analyze logon and logoff events. You can use the /n switch to specify the number of logon and logoff events that you want NTLast to report. You can use the /s and /f switches to report successful or failed logons, respectively; you can use the /r and /I switches to specify remote network or interactive (i.e., local) logons, respectively. When you select successful logons (i.e., when you use the /s switch), NTLast tries to find the corresponding logoff event for each logon, as Figure 4 shows. In this capacity, NTLast functions similarly to R528.

NTLast doesn't read archived .evt files, so you must use the tool's /m switch to run the utility against each remote system that you're monitoring. For example, to report the past 20 failed remote logons, type

ntlast /m oraclesvr /f /r /n 20

R592. In "Interpreting the NT Security Log," April 2000, I explain how to use the process tracking audit event category to audit programs that run on your workstations and servers. Security event ID 592 identifies a started process, as well as the process name, the user who started the process, and the date and time that the process started. Event ID 593 identifies an ended process. Event ID 592 and event ID 593 correspond in a manner similar to the complementary logon and logoff security event IDs 528 and 538. R592, another tool for purchase from Frank Heyne Software, searches one or more event logs for each event ID 592 and corresponding event ID 593 and produces a useful report showing who ran each process and how long the process ran. The source file and output file options are similar to R528's options. Although process tracking is valuable (especially on workstations), it has limits: Users can copy and change program names, and NT reports only the executable name rather than the full pathname.

System-log tools. When you use a RAS server, attackers can easily target this important entry to your network. Frank Heyne Software's R20050 provides RAS connection reports (RAS connections trigger System event ID 20048 and System event ID 20050). Cut from the same mold as R528 and R592, this utility offers the same options for source and output formatting.

Creating Custom Reports
Although the tools I've described can help you analyze the event logs and generate specialized reports, these tools deal with only certain activities. When you want to monitor other activities (e.g., file and object access, use of user rights, account management), you can use ELDump and Access to import the necessary event logs from all your systems into one Access database, then create custom reports that draw information from all the event logs. By using ELDump in combination with Access, you can overcome the inherent problems that arise from the different number of strings in each event ID's free-form description.

First, create an Access table that has columns for all the standard description data (e.g., date, time, event ID, username, source, type, category, computer name). Then, add 20 more text columns and label them with generic names (e.g., string1, string2) for the message strings. These extra columns will provide room for events with large descriptions. Create a batch file that uses ELDump to dump the Security logs for all your relevant systems into one text file. Use ELDump's -A parameter to specify output for only the past 24 hours of activity (i.e., -A 24). Use the -s parameter to specify each computer from which you want to dump the logs. For example,

eldump -s oraclesvr -s docsvr1 -s docsvr2 -A 24 -t -l sec > events.txt 

will dump the past 24 hours of activity from the Security logs for the servers OracleSvr, DocSvr1, and DocSvr2 into a tab-delimited file called events.txt.

Next, create a macro that automatically imports the text file into your Access table. Add a command to the batch file to execute the macro after ELDump runs. Configure the batch file to delete the text file after importing it into the Access table. Finally, schedule the batch file to run each day at the same time. Now you have a master database of all security events for your network, and you can write custom reports from this database. For example, you can write a Daily High Risk Events report that lists all the highly suspicious events that I described in "Protecting the NT Security Log," July 2000. You can sort the events by event ID, then by computer. You can also use this database for ad hoc queries when you need to research trends or investigate other situations. With this method, you might miss or duplicate some events depending on how exact your job scheduler is and how quickly the dump executes. If this possibility concerns you, you can insert a step in the batch file to first use EventSave to dump the logs, then use ELDump to read the archived .evt files rather than pull the events directly from your systems' logs.

In the Know
You'll need to dedicate time to set up regular automated archiving and reporting processes, but you'll save time in the long run by eliminating the need to manually peruse each event log. And after you've initiated these processes, the sky is the limit for your archiving and analyzing options. Take the time now to implement automated processes, and you can sleep easier knowing what is happening on your network.