Writing a Secure POP3 Server

By now, most users are probably familiar with POP3, a simple post office protocol for retrieving email from a mail server, often over the Internet. But how do you protect your POP3 server against attacks? For starters, you need to consider the various security issues involved in creating a POP3 server. Part of writing secure code is designing secure code and considering security issues before you start. I can't reiterate enough the importance of proper design. Too many software projects don’t design for security from the ground up, and trying to add it later often results in large security holes or wasted effort ripping out systems and rewriting them.

Despite POP3's simplicity, a lot of security issues are worth examining. If you look in the archives of any good security mailing list, you’ll find references to several problems in POP3 servers. Let's start by thinking about the threats we'll face in designing our secure POP3 server.

Types of Threats
The most obvious problem when creating a POP3 server is that you must present a service to the Internet, and you can expect people to abuse your server in several ways, including buffer overflows. Complicated processing of user input is also a ripe area for a denial of service (DoS) attacks aimed at CPU starvation or memory utilization. Attackers will also try to break into your system at the lower levels with attacks such as SYN floods. They'll try to deny service by making multiple connections to your server and letting these connections time out, which can deplete your pool of worker threads. If the attackers do manage to overflow your buffers, you’ll prefer that the service be running as some other user than LocalSystem because running as a less privileged account would reduce the impact of the attack. Finally, outside attackers might try to crack your user’s passwords with password guessing attacks.

As if worrying about network-based attacks isn't enough, you must also protect your POP3 server against local attacks. You probably don’t want anyone to sit down at a console (or launch a Terminal Server session for that matter) and run applications that can steal other people’s mail. If I run the service under a certain user context, then what context do I choose? Should I use a local account or a domain-wide account? What are the security and functional implications of each? Where in the Registry do I place the configuration information so that people can’t tamper with it?

Let's take a closer look at each of these attacks. In future columns, we'll examine these problems in depth and evaluate the best solutions for each.

Abusive user input. When you create a POP3 server, you can expect abusive input from users, such as overly long commands. For example, the programmer assumes that because a password can only be a few characters long, the attacker might try to give the POP3 server a command that looks like

PASS aaaaaaaaa

If the server abruptly drops the connection, it might result in an overflow. To deal with this threat, you need to examine the structure of the POP commands and consider not supporting some of the optional commands, such as the AUTH command, which can take arbitrary input. This command can be very helpful in increasing the security of this protocol, so if you do choose to implement it, do so carefully. In addition, you need to ensure that neither commands or arguments can overflow buffers.

You also want to process all user input in one place and decide what to do if you see abusive input. Processing user input from a socket is trickier than processing input from other sources because network interruptions can cause timeouts. If you don’t handle these failures properly, an attacker can leverage this mistake into a DoS attack.

Resource starvation. Resource starvation is another form of attack that presents a difficult problem for POP3 servers, with solutions that all have compromises. I doubt we’ll come up with a perfect solution in this series of articles, but I hope I can show you ways to avoid some of the more obvious errors.

Protecting yourself against resource-starvation attacks also protects you against broken clients and might help you slow down password-guessing attacks. An example of a resource starvation attack is if I had only five threads to handle user connections, and I let one client keep all my worker threads busy at once. A variant of this type of attack is a situation in which I spawn a new thread or process for each request without limits, so an external attacker can run my system out of memory.

Insecure files at installation. A common problem with many applications is that developers don't properly secure the files in their applications upon installation. For example, if you create a new directory on a freshly formatted NTFS partition, that directory will be set to give full access permissions to "Everyone" on the system. This level of access isn’t appropriate for most applications. When you’re dealing with someone’s email, this problem becomes a serious consideration. The POP3 server must be able to read and write to everyone’s mail.

The same problem goes for configuration information in the Windows 2000 or Windows NT Registry. You need to either be sure that the area where you are creating directories or Registry keys has correct permissions you want to inherit, or that you apply correct permissions at install time.

When I created the NT version of Internet Security Systems' (ISS') Internet Scanner, I took the concept of securing files one step further and checked for correct permissions at startup. Writing code to apply access controls can be difficult, and many installation programs don’t provide convenient hooks to set these. You typically end up writing your own code to do this and calling the code from within your install application.

Problems with sockets. When you develop a secure POP3 server, you are not simply limited to just the problems associated with the file system. Sockets let users bind a new socket to an existing socket, and unless you protect yourself from this type of attack, a user can snoop your traffic or even impersonate your server. Anyone writing a sockets-based network server, of which POP3 servers are a subset, needs to understand how socket binding preferences work with Win2K and NT systems and how to write your code so that this type of attack doesn’t happen.

Service user context. Services, and the user context that a service runs under, are tricky propositions. Whether you use the local system account, a domain user account, or a local user account, all three provide risks and benefits. If you have made a conscious decision about how your service is going to run, you can educate your usersin terms of how to roll out your product in a secure manner. They would much rather have you correctly install your service at install time than to hire a security penetration testing team that tells them your service has just ruined the security of their network. Service user context decisions are an often-overlooked and very important aspect of network security.

Although the thought of developing a secure POP3 server seems easy, the security design issues we face prove that it's not so simple. Now that you know what issues you need to overcome, you can plan for them, and if necessary, accurately estimate the time you need to complete the project. Any non-trivial application should face a security design review, and the best case would be a security-focused code review. As I said in my first article, be lazy and do it right the first time.