How to Set Up a DMZ with ISA Server

Many organizations don't implement a demilitarized zone (DMZ) in their networks. Instead, they locate their public servers (e.g., Web servers) on the same internal network as the rest of the organization's servers and workstations. Without a DMZ to separate your publicly accessible servers from your internal LAN, you're exposing your internal network to added risk. When an attacker gains control of your Web server, that person can use it to attack sensitive resources (e.g., financial applications, file servers) in your internal network. Notice that I said when and not if. No matter how securely you lock down a Web server, you should count on its compromise and design your network and processes to minimize damage and ensure their quick restoration. One such strategy is compartmentalization, and one tactical component of compartmentalization is implementing a DMZ.

When you implement a DMZ, you create two physically separate networks: one network for public servers and one network for all internal servers and workstations. Depending on the type of DMZ, one or more firewalls enforce routing policies specific to each network and strictly control access between

The main advantage of implementing a DMZ instead of just using a firewall is that when an attacker compromises a public server, the risk to the internal servers is reduced because the public servers and internal servers are separate from each other. When the compromised server resides in a DMZ, the attacker isn't able to directly attack other more-sensitive servers on the internal network. The firewall blocks any attempt by the DMZ computers to connect to internal computers, except for specifically allowed connections. For example, you might configure your firewall to let the Web server in the DMZ connect to an internal Microsoft SQL Server machine through a specific TCP port. If an intruder compromises the Web server, he or she might be able to attack the SQL Server machine on that port. However, the intruder won't be able to attack the SQL Server machine's other services and ports or other computers on the internal network.

Implementing a DMZ has other advantages as well. Those advantages include the following:

Although the advantages are many, you might pay a performance penalty for putting a firewall between your public servers and the Internet. Depending on bandwidth, firewall hardware, traffic levels, and many other factors, you might not notice the difference. However, some high volume sites simply can't afford the performance penalty and must rely solely on hardening the Web server and passive network Intrusion Detection Systems (IDSs).

Now that you know the advantages and disadvantages of implementing a DMZ, let's look at the considerations involved in deciding which kind of DMZ to implement. In addition, you need to be aware of some significant technical caveats before you design a DMZ. In this discussion, I explore how to use Microsoft Internet Security and Acceleration (ISA) Server 2000 to design a DMZ, but you can extend the principles to other firewalls. If you're unfamiliar with ISA Server, see the sidebar "Installing and Using ISA Server as a Firewall" for an overview.

The Types of DMZs
DMZs come in two types: three homed and back to back. A three-homed DMZ, which Figure 1, page 12, shows, consists of one ISA Server computer (i.e., firewall) with three NICs. The NICs connect the firewall to the Internet, the DMZ network, and the internal network. If you have the budget for another server and ISA Server license, you can implement a back-to-back DMZ. In this type of DMZ, one ISA Server machine (i.e., outer firewall) connects the Internet to the DMZ and another ISA Server machine (inner firewall) connects the DMZ to the internal network, as Figure 2, page 13, shows. Because the back-to-back DMZ has two firewalls that an intruder must infiltrate, it provides a higher level of protection for your internal network than the three-homed DMZ. Other differences between the two DMZ styles lie in protection for the public servers and cost. Some technical problems with IP addressing and certificates might also be factor in your decision of which style to choose.

The Three-Homed DMZ
The three-homed DMZ is the least expensive to implement because you need only one server, one Win2K license, one ISA Server license, three NICs, a hub or switch for the DMZ segment, and possibly public IP addresses for the servers in the DMZ. However, ISA Server has some important limitations associated with implementing a three-homed DMZ. ISA Server lets you designate only one network as an internal network with full protection. In addition, the three-homed DMZ provides application-level inspection for only the computers in the internal network. In other words, ISA Server's features for application-level inspection of HTTP, FTP, SMTP, POP3, and Microsoft Exchange Server remote procedure calls (RPCs) aren't available for computers in the three-homed DMZ. ISA Server offers only classic packet-filtering features for computers in the DMZ.

You designate the internal network by entering the appropriate IP address ranges in ISA Server's Local Address Table (LAT). Because ISA Server lets you enter only one LAT, you can only provide full protection to just one network—your internal network.

Another ISA Server limitation with implementing a three-homed DMZ is that ISA Server supports only IP packet filtering for computers in the DMZ. Consequently, you don't get the security benefits of Network Address Translation (NAT) or ISA Server's server-publishing, application-filtering, and client-access control features. This limitation also means you must assign valid public IP addresses to servers in the DMZ instead of using private addresses and NAT. So, if you're transforming a simple network to a three-homed DMZ, you might need to purchase additional IP addresses from your ISP.

The public IP addresses you assign to computers in the DMZ must be a subnet of your firewall's IP address. You can use an address for your DMZ segment that isn't a subnet of your firewall's address, but you need to notify your ISP so that it can route packets for your DMZ to your firewall. For example, suppose that your ISP provides you with two subnets: w.x.y.z/24, which is a public IP address that's part of the ISP's larger subnet, and a.b.c.d/29. You can assign w.x.y.z/24 to the external interface of your firewall. You can then have your ISP route all packets destined for a.b.c.d/29 to w.x.y.z/24, even though a.b.c.d/29 isn't a subnet of w.x.y.z/24.

To set up a three-homed DMZ that uses ISA Server as the firewall, you need to first install three NICs on the server. Then, you must install Win2K and its latest service pack (at the time of this writing, Service Pack 3—SP3) and ISA Server and it latest service pack (at the time of this writing, SP1). Next, run Windows Update to install any post-SP3 hotfixes, and visit http://www.microsoft.com/isaserver/downloads/default.asp to install any ISA Server hotfixes that came out since the release of ISA Server SP1.

With the installations complete, you can configure ISA Server's LAT with your internal network's addresses. Don't include any addresses in your DMZ or any other addresses on the Internet. Next, enable packet filtering and IP routing. To control the internal network users' access to the Internet and DMZ, configure the appropriate site and content rules and protocol rules. If you're unfamiliar with how to configure ISA Server, see "ISA Server: Your Network's Lifeguard," October 2001, http://www.winnetmag.com, InstantDoc ID 22251.

To let users on the Internet access servers in your DMZ, you must create IP packet filters that open the appropriate ports for each server in the DMZ. For example, if you have a Web server and SMTP gateway in the DMZ, you need to create a filter that opens up TCP port 80 to your Web server (or TCP port 443 if you use HTTP Secure—HTTPS). You then need to create two filters that allow connections to incoming-destination TCP port 25 and from outgoing-destination TCP port 25 on your SMTP gateway.

Next, you need to enable SMTP to flow between your internal Exchange server (or other email server) on your internal network and the email gateway in the DMZ. To do this, you need to create a server-publishing rule that lets the gateway open SMTP connections to the internal Exchange server for incoming email messages and create a protocol rule that lets the Exchange server open SMTP connections to the gateway for outgoing email messages. If your Web server needs to access resources on an internal server such as a SQL Server machine, you need to also create a server-publishing rule that lets the Web server in the DMZ access the SQL Server machine. By creating server-publishing rules, you limit the risk to which your internal servers are exposed in case an attacker gains control of the Web server in the DMZ.

Even with the server-publishing rules and protocol rules, you still have some security holes associated with the internal servers. Fortunately, you can apply another layer of protection. For example, to mitigate the risks associated with an internal Exchange server, ISA Server includes an excellent application filter for SMTP that lets you lock down the connection from your email gateway to your Exchange server. You can configure the SMTP filter to block certain SMTP commands, attachments, and domains and even scan email messages for specific keywords. The SMTP filter enforces size constraints on each allowed SMTP command to help prevent SMTP-specific buffer-overflow attacks. Mitigating SQL Server­based risks is more difficult than mitigating Exchange risks because ISA Server doesn't have an application filter for SQL Server. However, you can make breaking into SQL Server as difficult as possible. For example, you can implement good password and lockout controls on your SQL Server machine, even though only the trusted Web server has access to it. Make sure your Web application protects the credentials it uses to connect to the SQL Server machine—in other words, don't hard-code the username and password into the application's source code. And make sure you limit the SQL Server account to the bare minimum authority it needs within SQL Server to accomplish its job. Don't let your Web application use a systems administrator (SA) account to connect to the SQL server. Instead, have the Web application use an account with limited permissions.

Finally, you need to consider whether and how you will implement HTTPS to protect private areas of your Web site. Implementing HTTPS in a three-homed DMZ is easy because ISA Server simply routes packets between the Internet and the DMZ. To implement HTTPS in a three-homed DMZ, you just request a server certificate from a public Certificate Authority (CA) and install that certificate on the Web server in the DMZ.

The Back-to-Back DMZ
With two firewalls, you can use private IP addresses and NAT to conceal both your DMZ and internal network. As Figure 2 shows, the outer firewall publishes the DMZ's Web server and SMTP gateway to the Internet. The inner firewall publishes the Exchange server to the SMTP gateway.

To set up a back-to-back DMZ that uses ISA Server for the two firewalls, you need to obtain two servers, two copies of Win2K, two copies of ISA Server, four NICs (two for each server), and a switch or hub for the DMZ. First, create the outer firewall by installing and updating Win2K and ISA Server on one server. In the Control Panel Network Connection applet, configure the Internet NIC on the outer firewall with all the IP addresses that your ISP provides. Select a private IP subnet such as 10.10.*.* for all the public servers in the DMZ. For example, assign 10.10.0.1 to the DMZ NIC on the outer firewall, 10.10.0.2 to the Web server, and 10.10.0.3 to the SMTP gateway. Assign this subnet (10.10.*.*) as the outer firewall's LAT. Configure computers in the DMZ to use the outer firewall as the default gateway.

Next, create the inner firewall by installing and updating Win2K and ISA Server on the second server. Assign 10.10.0.4 to the DMZ NIC on the inner firewall, and configure that NIC to use the outer firewall as its default gateway. Select a private IP subnet such as 10.20.*.* for the internal network. Assign the address 10.20.0.1 to the internal network NIC on the inner firewall. Configure internal computers to use the inner firewall as their default gateway. Configure the inner firewall's LAT with the IP address of the internal network (10.20.*.*), which is a separate network from the 10.10.*.* network of the DMZ.

After you've configured the private IP addresses, you can set up the server-publishing rules and protocol rules. To let clients on the Internet access the Web server, add a Web-publishing rule to the outer firewall that publishes the Web server. To let SMTP servers on the Internet forward email to the gateway, configure a server-publishing rule that redirects incoming SMTP connections initially sent to the outer firewall to the SMTP gateway in the DMZ. Then, on the inner firewall, add a server-publishing rule that publishes the Exchange server's SMTP service but limits client connections to the SMTP gateway (10.10.0.3). Next, configure the SMTP gateway to forward incoming email messages to the inner firewall, which will silently redirect them to the Exchange server. To allow outgoing email destined for someone on the Internet, configure a protocol rule and an IP packet filter to allow outgoing SMTP connections to the SMTP gateway. Limit those connections to only those that originate from the Exchange server. Configure Exchange to forward outgoing email messages to the SMTP gateway. Finally, if the Web server in the DMZ needs to access an internal SQL Server machine, create a server-publishing rule on the inner firewall that lets the Web server access the internal SQL Server machine on port 1433.

Implementing HTTPS in sensitive areas of your Web site in a back-to-back DMZ differs from implementing HTTPS in a three-homed DMZ. Instead of installing the server certificate from a public CA on the Web server, you must install the certificate on ISA Server. Consequently, when a client on the Internet accesses a secure area of your Web site, the client browser negotiates a Secure Sockets Layer (SSL) connection between the client's computer and ISA Server. ISA Server then redirects the request to the Web server in the DMZ. You can configure ISA Server to either redirect the request as clear-text HTTP or establish a new HTTPS connection to the Web server. If your Web server is the only server in the DMZ, I recommend that you use HTTP to conserve computing resources. You'll have little risk of anyone intercepting communications because your Web server is on a trusted DMZ network. However, if the Web server isn't the only server in the DMZ, you might consider establishing a new HTTPS connection. Be aware that the reencryption process will reduce ISA Server's and the Web server's performance. Another option would be to use Virtual LANs (VLANs) on the switch for the DMZ to isolate traffic between ISA Server and the Web server.

This LAN Is Your LAN—Secure It!
A three-homed DMZ provides security for the internal network as well as some protection for the public servers in the DMZ. In addition, a three-homed DMZ is less expensive and easier to implement than a back-to-back DMZ. However, with a back-to-back DMZ, you get to use all of ISA Server's features to protect both the internal network and the public servers in the DMZ. In a back-to-back DMZ, the outer firewall inspects traffic to and from your Web server and SMTP gateway at the application level, whereas in a three-homed DMZ, all you get is IP address and port filtering. In addition, a back-to-back DMZ requires only one public Internet address and conceals the IP addresses of not just the internal network but also the servers in the DMZ.

No matter what type of DMZ you decide to implement, you're taking an important step in securing your LAN. Putting your publicly accessible servers in a DMZ adds another layer of protection against attacks.