Whitelist Script Cuts Off Spam

I wanted a way to create a whitelist that we could use with Exchange Intelligent Message Filter (IMF) on Exchange Server 2003 Service Pack 2 (SP2) without also losing my ability to use a blacklist. But if you convert the blacklist to a whitelist, you lose your blacklist functionality. To use the blacklist as a whitelist, the only method I could find through an Internet search was a solution (see http://forums.msexchange.org/m_1800395576/printable.htm) that causes items to be moved into the pickup folder where they're rerouted for delivery, whereas with traditional blacklisting, a piece of blacklisted mail is moved out of the delivery queue. Here are the steps for this solution:

This fix caused some problems, though. First, our corporate email servers would try to deliver duplicate messages to To: and CC: recipients that weren't part of our Exchange organization. And of course, this method eliminates my blacklist on which I depend to delete all those double–opt-in messages (i.e., sent to employees who have left the company or to employees who signed up for a nonapproved email subscription) that are hard to get rid of and very annoying. Lowering the spam confidence level (SCL) in IMF to a 1 reduced spam to less than one percent; however, lowering the SCL to 1 also resulted in a high number of false positives.

A Better Method
With the problem of false positives in mind, I wanted a way to filter the ExchsrvrMailroot vsi n UCEArchive directory that IMF generates to hold messages that meet the SCL threshold. I played around with a couple of command-line options and finally came up with the idea of writing a script that uses a whitelist to filter valid email from the UceArchive directory. The script reads the whitelist (a text file) and validates messages by comparing the email addresses in the UceArchive directory with those in the whitelist. Then the script moves the valid email from the UceArchive directory to the Pickup directory and moves spam email from the UceArchive directory to a designated folder.

This solution offered the best of all worlds:
• I could lower the SCL to 1.
• The script would resend false positives to the Pickup directory.
• I got to keep my blacklist for all those double opt-ins.

Here's how to use my script-based solution:

In WList.bat, you can modify the number of seconds the program waits for the loop to start each time to meet your company’s service level agreement (SLA) for email delivery. Additionally, you can further filter the Blist directory by filtering on the SCL rating itself. This appears as ‘X-SCL: #’, where the # symbol is the SCL rating number.

WList.bat runs quickly. In initial testing, the script scanned more than 1,200 email messages and moved them in less than three seconds using a whitelist consisting of about 200 line items.

Final Thoughts
Because the server-side SCL rating can't be lower than the email-client–side SCL rating, lowering the rating to a 1 also lowers it on Microsoft Office Outlook 2003 clients. This lower rating might cause some false positives on the client side. However, these false positives will soon disappear as users approve domains and senders. You can avoid this problem altogether by only enabling IMF on an SMTP gateway. Then redirect all port 25 traffic to the gateway and let it handle all the processing.

After extensively testing my script on a Friday, I put it into production on our Exchange server. That following Monday morning, our CEO told my director that he thought our email was down because he received no spam on his BlackBerry device! Before we started using WList.bat, about 10 to 12 percent of spam sent to users' accounts made it into their mailboxes. Now employees get no spam at all because our spam-filtering method is based exclusively on whitelisting.