Removing Duplicate Messages with Third Party Solutions

A while back, I logged onto one of my POP3 accounts and my Microsoft Office Outlook 2007 installation decided that I hadn’t downloaded a couple thousand messages that were already downloaded but left on the server. Outlook has an option to leave messages on the server for POP3 accounts. These were downloaded over the period of a month or so and each POP3 session completed with the QUIT command as outlined in the POP3 RFC. As they were downloading, had I stopped the transfer the POP3 session would not have completed properly and the downloaded status of the messages on the server would not have been committed. POP3 servers need the QUIT command from the client to commit updates on the server.

So after downloading the messages again, I now have a PST file with almost 2000 duplicate messages. There is no simple native way to remove the duplicates in Outlook without some coding. There are numerous third-party applications that can parse folders and remove duplicates. The Internet headers provide the most accurate assessment of duplication with the Message ID, but tools can easily compare other fields, including Sender, Date Sent (PR_CLIENT_SUBMIT_TIME), Recipients, and File Attachment properties.

The MAPILabs Duplicate E-mail Remover for Outlook tool works well to solve this issue. It removed over 1600 messages from my POP3 account PST file, as you can see in Figure 1. This would have been a challenging manual process otherwise. There are other third-party applications that also perform duplicate item removal from Outlook Data Files, including Outlook Duplicate Items Remover from Vaita and Duplicate Killer by 4Team. These solutions work through Outlook on POP accounts, IMAP accounts, and Exchange accounts.