Toolbox: Tail

Last month in Toolbox, we examined how the ubiquitous grep program can help you ferret out information from your log files or other text-based files (see "Toolbox: Grep," August 2005, InstantDoc ID 46869). As I mention in that article, another useful UNIX tool that's worthy of inclusion in your Windows toolbox is the tail program. In essence, tail displays the last few lines of a text file—especially helpful when you're examining log files. For example, suppose you're programming new rules for your firewall. You can use tail on your log file to see the rules' effect.

Tail is available on most UNIX systems, or you can download a GNU-licensed Win32 version from the Sourceforge Web site (http://unxutils.sourceforge.net). Download the UnxUpdates.zip file, then extract tail.exe to your computer.

Using Tail Alone
Used alone, tail displays the last few lines of a text file. The utility also supports several parameters that you can use to customize how it displays information. One particularly useful option is the follow (-f) parameter, which you can use to continuously watch for and display changes to a text file. For example, the command

tail -f ex050410.log

will display the last 10 lines of the log file named ex050410.log and will monitor the file and display new entries as they're added. If the file is a Microsoft IIS Web service log and someone accesses the Web site, IIS will write out a new log record. The new additions will immediately show up in the console that's running tail. This parameter eases troubleshooting by letting you immediately see new entries.

Using Tail and Grep Together
As I explain in "Toolbox: Grep," grep is a program that searches target text files for a specified pattern. For example, suppose you're troubleshooting a computer that's running Windows Firewall and want to search the firewall log for all activity from a certain date. The log isn't separated by dates and is quite large.

First, you can use grep to extract the lines of data from March 7, 2005 into a new text file by using the command

grep "2005-03-07" p-firewall.log > 030705p-firewall.log

So what about tail? Well, you can use tail on the firewall logs to assist with troubleshooting or to watch attacks in real time. But you can also use tail in conjunction with grep to display only certain data.

First, configure your firewall to send its logs to a text file. Every UNIX system uses syslog as its event logging system; most commercial firewalls also support syslog. If you're running grep and tail on a UNIX system, configure your firewall to send its syslog data to the UNIX syslog host. Windows users can install and use a Windows-based syslog server. I recommend Kiwi Enterprises' Kiwi Syslog Daemon, a great tool that can save syslog data into a text file.

Then, construct a pattern based on the firewall vendor's log syntax. For example, say you're using a Cisco PIX firewall and want to be notified every time someone accesses a Web service across the firewall. You can use tail and grep to scan the logs in real time for "/80" (which represents Web traffic in a PIX log), like so:

tail -f pix.log | grep "/80"

A savvier pattern uses regular expression metacharacters, which support more-sophisticated filtering than regular text strings do:

tail -f pix.log | grep /80space:

Web Figure 1 (http://www.windowsitpro.com/windowssecurity, InstantDoc ID 47176) shows the results of this second command. You'll find that mastering regular expressions takes time, but the payoff—a library of useful and effective patterns that you can use to search for just about anything—is definitely worth the effort.

A More Sophisticated Tail
Grep and tail are easy-to-use and highly flexible programs. When you work with console applications, both tools will greatly aid your log file analyses and day-to-day systems administration. The command-line version of tail is fast and easy to use, and purists will probably prefer it because of its simplicity and ability to pipe output into other programs such as grep. But you can find Windows GUI versions of tail, some of which offer more sophisticated features such as color highlighting of matched patterns. Such formatting features can help you keep tabs on important files.

An example of a free Windows GUI tail program is Bare Metal Software's BareTail utility (you can download the program at http://www.baremetalsoft.com/baretail). Like tail, BareTail displays a text file and follows new additions to the file, but because BareTail runs in a GUI, it offers highlighting features, as Web Figure 2 shows. This capability makes it easier to spot certain text (e.g., a specific IP address or port) on-the-fly when watching a firewall log. You can also change the font, easily copy a line of text, and open the most recently viewed log files by using Windows' most recently used (MRU) list. If you have specific needs for tail, search the Internet and you'll find many customizations of the tool.