How to Efficiently Search and Manage Event Log Data

It’s no secret that checking event logs is tedious. I’d like to show you some techniques you can use and new technologies available in the Windows Server 2008 Event Viewer that let you zero in on specific events of interest.

Rather than having to check the contents of each different log, you can configure Windows to alert you when something interesting has happened. Plus you can efficiently search and present event log data so you can get on with the rest of the tasks that take up your day.

Know What to Look For

Searching through event logs for evidence of a particular event can be like looking for a needle in an event log haystack, especially if it’s not immediately obvious whether you should be checking fields such as String, Messages, Data, EventType, EventTypeName, EventCategory or EventCategoryName. You might suspect that an event has occurred and has even been logged, but you might not be sure what evidence exists of that event within the event log.

The key to effectively examining event logs is knowing what to look for. Rather than searching all event log fields using key terms, searching for specific events by their event ID is a more effective way of locating evidence.

The catch is that you have to know what event ID correlates to a specific event. The event IDs in a Server 2008 event log are all well documented, either on TechNet or elsewhere. I’ll cover a few event IDs of interest a little later in this article.

IT pros should note that Server 2008 event IDs use a different numbering system to those used in earlier versions of Windows. For example, an account lockout is recorded as event ID 644 in Windows 2000 and Windows Server 2003 event logs, but event ID 4740 records account lockouts on Server 2008.

This new numbering system means that existing scripts that trawl logs for event IDs related to events in Windows 2003, Windows XP and Win2K won’t work when you run them against the logs on a computer running Server 2008. Some notable Server 2008 security event IDs are listed in Table 1.

These events are primarily security related. Some of them might be entirely benign, but if any of them occur with great frequency in your environment, it might be worth investigating further. You can find a list of all Server 2008 security-related event IDs at http://support.microsoft.com/kb/947226.

Filtering and Custom Views

Filters are quick-use tools that let you limit the displayed data in a single log. Filters in Server 2008 work mostly the same way as they did in Windows 2003.

Although filters aren’t persistent, you can save a filter as a custom view. To create a filter on a Server 2008 computer, perform the following steps:

The drawback to filtering on the basis of event ID is that you need to know the ID of the event that you are looking for. Searching by user or computer doesn’t return all events relating to that user or computer. For example, if an account logon event is associated with the User: “N/A”, it is only by looking within the EventData field that you’ll actually see the user name that was logged on. You can’t enter freeform keywords, but can only select from a list of event-related key words.

A custom view is essentially a filter that you can reuse and apply to multiple event logs. Custom views are new to Server 2008.

As you can see in Figure 2 where a custom view has been created to show all events related to ID 4738, custom views get their own node within the Server 2008 event viewer. To create a custom view, perform the following steps:

Another useful feature of custom views is that you can export them, then import them on other Server 2008 computers. This lets you better manage the time you spend going through logs. Rather than scanning the logs for specific events, you can configure a custom view to present all interesting events that have occurred on a computer in a single location. This is because unlike filters, custom views can apply to multiple logs. 

After you set up an effective custom view, you can then place that custom view on each computer where you need to regularly review the event logs. Rather than having to trawl through every log on the server, you only have to look at a single list as the custom view will contain all events that you have defined as interesting. Custom views can function as a single port-of call, ensuring that you don’t miss an important event.

Scanning Logs from the Command Line

Although custom views and filters can provide you with a list of interesting events, you can’t use these tools to summarize the information contained within event logs. You can use a custom view to see all of the failed logon events, but you need to use other tools to summarize failed logon events on the basis of user account.

Perhaps the most famous Windows log tool is Log Parser, which can be downloaded for free at http://www.microsoft.com/DownLoads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en. Log Parser lets you query event logs using SQL syntax. For example, the following Log Parser command provides you with a list of accounts that have been added to the Administrators, Domain Admins, or Enterprise Admins groups:

logparser –i:evt "select extract_token    (Strings,0,'|') from Security where    EventID IN (4780)"

Although Log Parser works well with Server 2008, it hasn’t been updated since January 2005. This suggests that support for the tool may be deprecated in favor of Windows PowerShell.

It’s possible to write scripts in PowerShell that accomplish exactly the same things that you can accomplish using Log Parser. Unfortunately, as amazing as PowerShell is, unless you are comfortable with it, you won’t find its syntax as intuitive as Log Parser. The following PowerShell command, executed from an elevated PowerShell prompt, also retrieves data related to event ID 4780:

Get-Eventlog -logname Security | where    {$_.EventID -eq "4780"} | Format-Table    –Wrap TimeWritten,Message

Finally, you can also use the wevtutil.exe command-line utility to examine event logs. Wevtutil.exe can be very useful on Server 2008 Server Core computers that don’t support PowerShell. The following command locates events with event ID 4780 in the security log:

Wevtutil qe Security /rd:true /f:text /q:     *[Security[(EventID=4780)]]

The biggest drawback to the wevtutil.exe utility is the lack of documentation on how to use it. It also can’t be configured to collate data, unlike PowerShell and Log Parser, which can. I include Wevtutil here only for the sake of completeness.

Event Forwarding

Event forwarding is a new Server 2008 feature that allows events to be forwarded from one Server 2008 computer to another over the HTTP protocol. The advantage of this is that you can set up the equivalent of a log collection server, so rather than having to check the event logs of each computer on the network for specific events, you can check for events from many computers in a single place.

Consider account logon events. Any domain controller (DC) in your organization can authenticate a particular user’s logon. To determine which DC authenticated a logon, you check the security logs of each DC in your domain, though the DC that authenticates a logon is almost always located at the same site as the client logging on.

With event forwarding, you can forward all events, or just specific events, which you can define using a collection filter, to a central computer called a collector. Rather than having to check each DC in the domain, you just check the collector computer that has copies of the account logon events from all DCs in the domain. Server 2008 lets you configure two types of event log subscriptions:

Collector-Initiated Subscription. With this subscription type, a central computer polls a set of source computers to retrieve event log data. Collector-initiated subscriptions require manual configuration on each source computer as well as the collector computer.

This involves running winrm quickconfig from an elevated command prompt at each source computer, which allows remote management and configures a firewall exception. You also need to add the computer account of the collector computer to the local Administrators group on each source computer. I cover configuring a collector computer later in this section.

Source Initiated Subscription. When you configure a source-initiated subscription, each computer forwards events to a collector computer. You can configure source-initiated subscriptions using Group Policy. You do this by editing the “Configure the Server Address, Refresh Interval, And Issuer Certificate” policy located under the Computer ConfigurationPoliciesAdministrative TemplatesWindows ComponentsEvent Forwarding node.

Configuring the collector computer is similar for both the collector-initiated subscription and source-initiated subscription methods. To do this, perform the following steps:
 

Use the Select Events button and perform a process identical to creating a custom view to select the types of events the collector computer gathers or forwards. The default settings are for the collector computer to place forwarded events into the Forwarded Events log, though you can configure a different destination instead. You can apply custom views, filters, or scan the Forwarded Events log using Log Parser or the Get-EventLog function of PowerShell.

Getting Alerted

Event viewer tasks lets you start a program or send a message or an email whenever a particular event occurs, and that event is logged to the Server 2008 event logs, including an event written to the Forwarded Events log on a collector computer. Configuring such a task ensures that you are made aware of the event at the time it occurs, not when you get a chance to review the event logs later. Event viewer tasks are similar to the alert triggers that are available in Windows 2003 but simpler to configure, because in Windows 2003 you had to configure triggers from the command line using the eventtriggers utility.

In Server 2008 you can create an event trigger directly from event viewer by right-clicking an event and selecting Attach Task To This Event. This launches the Create A Basic Task Wizard where you specify what action you want Windows to take when a new event that has this event ID is logged. You can also create an event viewer task using the Task Scheduler console:

Event viewer tasks can be imported and exported, so you can deploy them easily on multiple servers throughout your organization. You remove event viewer tasks through the Task Scheduler console and not through event viewer.

You should be careful in selecting which event viewer tasks you set up. If you configure too many event viewer tasks to alert you, you will be deluged with notifications and will most likely begin to ignore them. You should choose events such as event ID 4780 (see Table 1), which happen rarely but are important enough to demand your attention.

Limiting Data

The key to dealing with event logs is being able to zero in directly on the data that is of interest to you. I discussed how you can view only interesting events using filters and custom views, how you can search logs using Log Parser and PowerShell, how you can centrally collect logs using event log forwarding, and how you can be notified as soon as an interesting event occurs by using event viewer tasks. In combination, all of these techniques can reduce the area of haystack that you have to deal with, making it a lot simpler to locate relevant needles.