Hunting Malicious Code

An antivirus expert outlines 7 steps for identifying viruses and Trojan horses on your system.

ITPro Today

May 19, 2002

15 Min Read
ITPro Today logo in a gray background | ITPro Today

7 steps to identifying viruses and Trojan horses on your system

The phone calls always start the same way: "My antivirus scanner isn't finding anything, but I know something is there." People don't call an antivirus consultant until the usual antivirus tools and checks have failed. And the callers' statement doesn't surprise me. I have more than 20,000 different malicious computer programs in my collection (not even half of what's out there), and hundreds of them aren't detectable by any scanner. Only a few byte changes or a different encryption scheme can make a familiar program unrecognizable. Even the best virus scanners can't find everything.

So, how do you find malicious code (e.g., worms, viruses, Trojan horses, backdoor programs) when the expert tools can't find it? Seven steps will help you find viruses and other types of malicious programs on all Windows systems. (I don't cover macro- or boot-virus detection. Describing the techniques for finding these two types of malicious mobile code would significantly lengthen this article, and boot viruses aren't as popular as they used to be.)

1. Disconnect from the Internet
If you have a live connection to the Internet, physically unplug it. A rogue program on your system might be in the middle of downloading data or placing more bad programs. If intruders have placed a remote access Trojan horse on your PC so that they can watch your every move on the system, you want to cut off their backdoor access before they realize that you're on to them. Many rogue programs have "kill" batch files that erase evidence or contain destructive payloads that crackers can trigger if they notice your discovery search. Windows XP and Windows 2000 let you disable network connections from within My Network Places, but sometimes you remain connected when you think you aren't. Physically disconnecting is the safest way to proceed. Remain disconnected until you've cleaned your system or confirmed the absence of harmful code.

2. Rerun Your Antivirus Scanner
You've already run your antivirus scanner once, but run it again. Before you run it the second time, make sure all other programs are shut down. Close programs in the System tray (you can usually do this by right-clicking the program icons, then clicking Close or Exit). Press Ctrl+Alt+Del to start Task Manager and close other unneeded programs. Closing programs, then running a virus scanner a second time are a good idea even when the tool finds infected files on the first pass. Many times, I've used scanners and found multiple infected files. Then when I've run them again with fewer programs in memory, I've found many more infected files than the first time. In general, the fewer things running in memory, the more accurate a scanner can be. Close everything that isn't essential. Notice that I don't recommend booting into Safe mode. Booting into Safe mode often disables Windows-based antivirus scanners or complicates the detection process later on.

Also be sure to turn on your scanner's most invasive scanning mode. You might have done your initial virus scan with the default settings that the vendor configured for typical, day-to-day use. The vendor has made cost/benefit decisions in the default settings so that the tool will complete scans more quickly by skipping the search for less frequently occurring malicious file types. For example, many scanners don't scan the Recycle Bin folder, a fact that malicious coders have used to their advantage. But if you're suspicious, you want the scanner to look for everything, everywhere, even in compressed files.

3. Look in Startup Areas for Suspicious Programs
Most intruders want their viruses and Trojan horses to be loaded into system memory as frequently as possible. A malicious program can place itself in dozens of spots so that it's automatically executed the next time the PC or some other program starts. You need to search each location and rule out every program that's automatically launched. This step is probably the most difficult and time-consuming part of the rogue-code hunt because finding malicious code means knowing which programs are legitimate. You might have to do quite a bit of research to rule out all the nonmalicious programs.

Each time you start your PC, it goes through a series of steps to load the OS and many autostart programs. Before you start one program, your PC might have already called a dozen programs and hundreds of files that the OS and other software need. Microsoft Task Scheduler, AOL, RealNetworks' RealPlayer, and video-card programs are commonly loaded programs. Many different areas of Windows load programs and files. Figure 1 shows some of the places, in order, that can load programs during a generic startup of a Windows Me or Windows 9x machine. XP, Win2K, and Windows NT follow a different startup path. Figure 2, page 90, shows the generic startup path for XP, Win2K, and NT.

The following files and areas can automatically execute malicious code. At a minimum, you should check these components and note any statements that load programs whose names you don't recognize. Later in this article, I explain how to check the legitimacy (or illegitimacy) of any unfamiliar or suspicious programs.

Config.sys or config.nt. Windows Me and Win9x's config.sys file configures the operating environment (e.g., specifying how many files the system can have open at one time and how much space stacks and buffers can use) and loads device drivers. Although intruders don't use config.sys as a launching pad as often as they do other Windows areas, config.sys can load malicious code as a driver or executable file. Pay particular attention to config.sys DEVICE= statements, which load device drivers, and SHELL= statements, which tell earlier versions of Windows which DOS-interface shell to load. If a DEVICE= statement names a file without a path, the device driver is in one of the Windows system areas, such as the root directory, Windows directory, or System directory. Some malicious programs create Trojan horse files with names identical to real Windows system files but locate them in atypical places. For example, DEVICE=IFSHLP.SYS is a legitimate file; DEVICE=C:WINDOWSSYSTEMIFSHLP.SYS probably isn't. Put on your list of suspicious programs any unfamiliar driver names or familiar driver names that are in the wrong place. SHELL= statements should usually point to command.com, although they might point to other legitimate interface shell programs.

Windows Me and Win9x execute some basic config.sys files and statements even if a config.sys file doesn't exist. (Remember this small fact so that you don't get too worried when files that aren't listed in your config.sys file are being loaded into memory and commands that aren't listed in config.sys are running.) The io.sys file in these Windows versions stores the following config.sys files and default settings: himem.sys, ifshlp.sys, setver.exe, dblspace.bin or drvspace.bin, DOS=HIGH, FILES=60, LASTDRIVE=Z, BUFFERS=30, STACKS=9,256, SHELL=C:COMMAND.COM /p, and FCBS=4. During a boot, the io.sys file checks for the presence of a config.sys file in the root directory. If io.sys finds config.sys, io.sys executes the statements found there. Then, io.sys processes any default settings stored in io.sys that config.sys didn't override. Thus, you might see config.sys statements during boot-up even if they don't exist in a config.sys file. You can't edit the default values in io.sys, but you can override them in config.sys.

Each new release of Windows relies less on the config.sys file, which is a holdover from DOS. In the 32-bit versions of Windows (XP, Win2K, and NT), config.sys is named config.nt and only older programs require it. If a config.nt file exists on your 32-bit system, check it as you would a config.sys file for statements that load unfamiliar programs.

Autoexec.bat or autoexec.nt. The autoexec.bat or autoexec.nt file, which runs after config.sys or config.nt, respectively, further manipulates the Windows environment (e.g., setting path statements) and loads more programs. You might also find separate autoexec.dos and config.dos files, which Windows loads when you run DOS programs in Windows Me or Win9x. Inspect your Autoexec file to make sure that you understand everything that it does and loads. Autoexec and Config often load CD-ROM and sound-card files. When malicious code adds itself to Autoexec, it typically does so at the end of the file, so read the file from bottom to top. Make a note of any suspicious files that you want to research later.

How do you know which statements an intruder might have added to your Autoexec and Config files recently and which statements were there originally? One way is to keep a copy of these files in a backup directory. Then, you can compare the current copies with your backup copies to see whether someone has modified the files. Another way, albeit less reliable, is to check the current Autoexec and Config files against any autoexec.bak and config.bak files, respectively, that you might have on your system.

Winstart.bat. If you find a winstart.bat or dosstart.bat file, check what it attempts to load. Win9x uses these files to load programs when Windows or DOS starts. For example, in Win98, if you open a DOS prompt, the system runs any commands in dosstart.bat. When you exit the DOS command shell back into Windows, winstart.bat is executed. Although malicious code rarely hides here, it can't hurt to inspect these files if you find them. List any programs that you don't recognize; I discuss how to research suspicious programs in Step 6. XP, Win2K, and NT don't use the winstart.bat and dosstart.bat files.

System.ini or ...Shell registry subkey. The [boot] section of the system.ini file in Windows Me and Win9x contains a SHELL=explorer.exe command, which specifies that Windows Explorer is the GUI file to load. Don't confuse explorer.exe with Microsoft Internet Explorer (IE—iexplore.exe). Viruses (and to a larger extent, Trojan horses) often modify the SHELL=explorer.exe statement so that they'll be loaded into memory along with Windows Explorer each time Windows starts. For example, in the statement SHELL=explorer.exe thing.exe, thing.exe is a piggybacking Trojan horse. The system.ini file also loads the screen saver. Intruders often alter the SCRNSAVE.EXE= statement to replace the name of the legitimate screen saver with the name of a malicious program. XP, Win2K, and NT load the GUI shell and screen saver from the HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsNTCurrentVersionWinlogonShell subkey instead of from the system.ini file.

Win.ini. Another initialization file Windows looks for during startup is win.ini, whose statements control the overall look and feel of the Windows environment. For our purposes, the [windows] sections of the win
.ini file are the most important. The LOAD= and RUN= statements automatically execute programs when Windows starts. Programs that LOAD= statements start run in a minimized state; programs that RUN= statements start run in a normal window. Viruses and Trojan horses can place themselves in these statements to take control whenever Windows starts. XP, Win2K, and NT don't depend on a win.ini file; later in this article, I discuss a registry subkey that's equivalent to the file.

Wininit.ini. Another .ini file rarely talked about is wininit.ini, which Windows and Windows programs use to install and uninstall programs. It can contain commands that copy or delete files. If a program places wininit.ini in the Windows folder, the system executes the file's instructions upon the next reboot, then typically renames the file to wininit.bak. At least a few malicious programs (e.g., W95.MTX, W32.Kriz, VBS.Kidarcade) use it to install themselves on Windows machines. If you see a wininit.bak file, open it with a text editor to see the last file operations wininit.ini performed.

If any of the previous files or areas (e.g., autoexec.bat, config.sys) contain a line with the name of a malicious program, you can delete or remark (REM) the line, save, and reboot the system. If you aren't sure whether a program is malicious, add it to your list of suspicious programs to research later.

The registry. Windows uses the registry to manipulate the OS and programs. You can use regedit.exe in Windows Me or Win9x and regedt32.exe in XP, Win2K, or NT to view and edit the registry, but if you do so, make sure you know what you're doing. One mistake can cause many more problems than what you wanted to fix.

When hunting for malicious code, you need to look at registry subkeys that automatically load programs when Windows starts. The subkeys that Figure 3 shows automatically load programs and run in the order listed. If you see a program in your registry startup areas that you know shouldn't be running every time you start Windows, you can right-click it, then click Delete. This action prevents the registry from autoloading the program, but you must also reboot the system to remove the program from current memory.

The Startup group. The Windows Startup group (Start, Programs, Startup) can store and start programs. In Windows Me and Win9x, use Windows Explorer to go to C:%windir%start menuprogramsstartup. In XP and Win2K, go to C:documents and settingsusernamestart menuprogramsstartup. And in NT, go to C:%windir%profilesusernamestart menuprogramsstartup. To delete a program, right-click it and click Delete.

4. Use Helpful Tools
Manually searching all your autoloading areas can be a chore, but Microsoft has provided several tools that can help. System Configuration Editor (sysedit.exe), System Configuration Utility (msconfig.exe), and Dr. Watson (drwatson.exe) are all great programs to run when looking in Windows files and areas for malicious code. Keep in mind that not all of these tools are available in all versions of Windows, and different versions of a tool can contain different capabilities.

Sysedit has been around since the Windows 3.x days, works with all Windows versions, and displays your config.sys, autoexec.bat, system.ini, and win.ini files, among others. As its name suggests, Sysedit lets you edit the file information it displays, although it doesn't display as much information as some of the other tools.

Msconfig, which is available in all current Windows versions except Win2K and NT, shows the same files as Sysedit, plus the programs being automatically run from the registry. You can use Msconfig to selectively turn autoloading on and off for programs. The utility's Startup tab shows programs run from the registry.

Dr. Watson is an excellent tool in Windows Me and Win98. After you start it, choose Advanced View to see the autoloading programs plus tons of other information (e.g., running OS files). Unfortunately, Dr. Watson is a view-only utility and doesn't let you disable autoloading.

5. Find Modified System Files and Executables
Although viruses and Trojan horses can easily hide the changes they make to your system, most don't because most people don't look for these changes. Malicious code often modifies program executables or adds rogue files. Thus, I usually use the Windows Find utility to look for all files modified in the last few days or since I first suspected a problem. Find always locates dozens to hundreds of files because Windows does a lot in the background. A lot of recently modified files with .dll, .exe, or .vxd extensions could mean you have some malicious code. Unless you recently installed a new program, you shouldn't have a lot of recently changed files with these extensions.

If you have a lot of these files, you might want to try an experiment. Run the suspected files to get them into memory, then run program files and executables that don't have a recent modification date. Then rerun the Find feature to see whether it locates more recently modified .dll, .exe, or .vxd files. If malicious code is present on your system, often it will infect more files as you execute them. If you end up with more modified files of these types, definitely suspect malicious code.

6. Research Suspicious Files
By now, you probably have quite a list of questionable files. You should have the filename, location, size, and creation or last-modification date. Now is the time to do some research. The subdirectory that a file is in can often give you a clue to the associated software product or vendor. You can try searching for the file on your installation disks (if you have any) or on the Web (using a PC different from the one you've disconnected from the Web for troubleshooting). I almost always do the latter. Type in the filename at a good Web search engine, such as AltaVista, Google, or Yahoo!. If the file does contain a malicious program, the search engine often returns links to malicious Web sites or cracker bulletin boards. If the file is legitimate, you'll see links to the vendor's Web site, at which you can compare your file's creation date and size with the vendor information.

Checking a file's date and size is important because a few Trojan files use a filename that's the same as or similar to a legitimate filename to fool users. Intruders use legitimate filenames for Trojan horse programs or overwrite legitimate programs with Trojan horse versions. They also use look-alike names, as Web Table 1 (http://www.winnetmag.com, InstantDoc ID 24899) shows. A filename that doesn't look quite right should set off warning bells.

7. Open Suspicious Files
If you can't identify a specific executable or program file through a Web search, try using the DOS edit.com program to open the file and visually inspect it. Although many malicious files are encrypted, they often contain words and phrases not found in most legitimate files (e.g., Virus!!, Infected!, Kill, Victim, Props to my hacker brothers). Web Figure 1 (http://www.winnetmag.com, InstantDoc ID 24899) shows a portion of the Hybris virus, which includes the sentence, "Greetz goes to ZOMBiE, Spanksa and Mister Sandman"—a sentence not likely to be in any file you should trust.

If all your files check out fine, you probably don't have a malicious code problem. Reenable your Internet connection, and troubleshoot as you would any typical software or hardware problem. If you do still have suspicious files after completing all these steps, consult your antivirus or security vendor (send the vendor a copy of the files), or consider how to treat a machine you consider unsafe. Most security experts would recommend a complete, clean reinstallation of the OS, all files, and programs. Happy hunting!

Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.

You May Also Like