IPsec for Network Protection

Three scenarios to improve the security of the hosts in your network

Steve Riley

April 25, 2005

12 Min Read
ITPro Today logo

Although typically used for VPNs, IP Security (IPsec) can do so much more to help you keep your network secure. You can use IPsec to solve three common problems—to stop worms, to protect servers, and to isolate a domain—and none of these techniques involves performance-sapping encryption. Let's look at the steps involved in each technique.

Using IPsec to Stop Worms
The best way to stop worms is to not get them in the first place. Alas, because some people don't understand or even care about the threats and risks associated with email and Web surfing, worms and other kinds of malicious code are a fact of life right now. Given this unfortunate situation, how can you reduce the damage that such code inflicts? You can thwart malicious code in three ways: prevent it from being installed, prevent the code from being executed, or prevent it from communicating.

In some instances, your only option might be to prevent malicious code from communicating. IPsec policies can help by limiting the kinds of traffic a computer accepts and generates. Rules with filter actions that specify simply to block or allow traffic (without building any IPsec security associations—i.e., authenticating or encrypting traffic) can act as effective basic packet filters on individual computers. Use Group Policy to assign these rules to computers, and you can reduce the amount of malicious traffic propagating throughout your network.

Your choice of IPsec policies depends on which OS you're running. Windows Server 2003 and Windows XP include Windows Firewall, which is more effective than IPsec for blocking inbound traffic, so if you run these OSs and are using Windows Firewall, your IPsec policies need to block only outbound traffic. Windows 2000 doesn't include a host-based firewall, so you should consider IPsec policies that block both inbound and outbound traffic for systems running Win2K.

Consider the Slammer worm. Slammer finds computers running Microsoft SQL Server or Microsoft SQL Server Desktop Engine (MSDE) by blasting the network with messages to UDP port 1434. Microsoft released a patch for this worm, but patching all computers can take some time; an excellent interim mitigation is to use Group Policy to quickly assign an IPsec policy that blocks inbound traffic to the vulnerable port. Of course, because the policy blocks inbound traffic to your SQL Server systems, don't leave this policy enabled on these systems after you've patched them.

To prevent a computer from getting infected by Slammer, assign a policy that blocks all inbound traffic from anywhere to the computer's own IP address with destination port UDP 1434:

  • Filter list with one filter: from any-address:any-port to my-address:1434/udp

  • Filter action: block

  • Rule: link the list with the action; all interfaces; no tunnel; any authentication method (the method doesn't matter because a block filter doesn't have any IPsec security associations)

To create the policy, first open the IPsec management console on the computer you want to protect (the process is the same for Windows 2003, XP or Win2K):

  1. Double-click Local Security Policy in the Administrative Tools folder.

  2. Click IP Security Policies on Local Computer.

Now create the filter list:

  1. Right-click in the right-hand pane of the Local Security Settings window, and select the Manage IP filter lists and filter actions menu option.

  2. On the Manage IP Filter Lists tab, click Add.

  3. Enter Slammer filter list for the name.

  4. Click Add, then click Next at the first IP filter wizard screen.

  5. Select Any IP Address for the source.

  6. Select My IP Address for the destination.

  7. Select UDP for the protocol.

  8. Click the To this port option, and enter 1434.

  9. Click Finish to end the wizard.

  10. Click OK.

Now create the filter action (if you already have an action called Block, skip this part):

  1. On the Manage IP filter lists and filter actions dialog box's Manage Filter Actions tab, click Add, then click Next at the first IP Security filter action wizard screen.

  2. Enter Block for the name.

  3. Select Block for the behavior.

  4. Click Finish to end the wizard.

  5. Click Close to finish setting up the filter list and action.

Now create the IPsec policy:

  1. Right-click in the right-hand pane of the Local Security Settings window, select Create IP Security Policy, and click Next at the first screen of the IP Security policy wizard.

  2. Enter Slammer filter for the name.

  3. Clear the Activate the default response rule option, and click Next.

  4. Leave the Edit properties option selected, and click Finish to end the wizard.

Now add the rule to the policy:

  1. The policy's properties dialog box appears. Click Add, then click Next.

  2. Leave the next three wizard screens at their defaults.

  3. From the list of filter lists, select the Slammer filter list.

  4. From the list of filter actions, select Block.

  5. Click Finish to end the wizard, and click OK to close the rule properties dialog box.

  6. Click Close to close the policy properties dialog box.

Now assign the policy:

Right-click the Slammer filter policy and select Assign.

Scripting IPsec Policy Creation
You can also use a command-line tool to create IPsec policies—this capability is useful for scripting. For Win2K, the tool is ipsecpol.exe from the Microsoft Windows 2000 Resource Kits; for XP, it's ipseccmd.exe from the Windows Support Tools for Microsoft Windows XP; for Windows 2003, it's the Netsh Ipsec tool included with the OS. To apply the Slammer filter in XP, use the following command:

ipseccmd -w REG -p "Block UDP 1434 Filter" -r "Block Inbound UDP 1434 Rule" -f *=0:1434:UDP -n BLOCK -x

Be sure to use uppercase and lowercase letters as shown; Ipsecpol and Ipseccmd are picky about case. Also note that this command is a single command that should be typed on one line—it's wrapped here to fit the article's column format.

This command creates and assigns a static policy called Block UDP 1434 Filter with a single rule called Block Inbound UDP 1434 Rule that contains the same filter list as above linked to a block filter action. Static policies are stored in the registry and persist between reboots. The policy won't apply until the next boot or restart of the IPsec policy agent, so if you want the policy to be applied immediately, your script should also stop and restart the policyagent service. When you build the policy with the GUI, it's applied immediately.

If a computer does get infected with Slammer, you can use a different IPsec rule to prevent the computer from infecting other computers by blocking outbound communications to destination port UDP 1434:

  • Filter list with one filter: from my-address:any-port to any-address:1434/udp

  • Filter action: block

  • Rule: link the list with the action; all interfaces; no tunnel; any authentication method

Note the subtle difference here: In the earlier (inbound) rule, the filter checks traffic from any-address:any-port to my-address:1434/udp, whereas in the outbound rule, the filter checks traffic from my-address:any-port to any-address:1434/udp. The outbound rule blocks any traffic to UDP port 1434 on any computer. Use the following command to script the rule and add it to the same policy as the first rule:

ipseccmd -w REG -p "Block UDP 1434 Filter" -r "Block Outbound UDP 1434  Rule" -f 0=*:1434:UDP -n BLOCK

In the first command above, the -x switch indicates a new policy. In this second command, you omit the -x switch because this command is adding another rule to the existing policy. Note also that in the filter list portion of the second command, the asterisk (*) and the 0 are reversed from the first command; this is because the "direction" of the filter is reversed.

You can also use the command-line utilities to apply dynamic policies, which stay in effect only as long as the system is running (they're lost when the policyagent service is restarted or the computer is rebooted). Dynamic policies are useful when you know you won't need a policy for a long time and want to use a reboot to clear it. The following two commands create a dynamic policy that does the same thing as the static policy I created above:

ipseccmd -f[*=0:1434:UDP]ipseccmd -f[0=*:1434:UDP]

The square brackets that enclose the filter specification indicate that the filtered traffic will be blocked.

So far we've been exploring how to use IPsec to block traffic from and to known "bad" destination ports. You could be more restrictive with your IPsec policies—blocking all traffic in both directions and creating rules that permit certain traffic from and to certain ports. Think carefully about what kinds of traffic to allow, plan extensively, and thoroughly test your ideas before launching them into production.

Using IPsec to Protect Servers
One good use of a "block-all-except-for" policy is for server protection. Why should a Web server, for example, accept anything other than inbound Web traffic on its Internet-facing connection? You can use an IPsec policy to build a rudimentary packet filter that discards everything except whatever makes sense for the purpose of a particular server—in the case of a Web server, everything except traffic destined for TCP port 80 (and TCP port 443, if some pages use HTTP Secure—HTTPS).

You could also use a server-protection policy to buy time for testing and deploying patches. If someone discovers a vulnerability in the OS, you can assign a policy to deny access to the vulnerable service to give yourself additional time to test and deploy the patch during a scheduled outage, rather than violating a service-level agreement that might be in place.

Let's look at an IPsec policy for a Web server. The policy would have two rules:

Rule 1

  • Filter list with one filter: from any-address:any-port to my-address:any-port

  • Filter action: block

  • Rule: link the list with the action; all interfaces; no tunnel; any authentication method

Rule 2

  • Filter list with two filters: from : to :80/tcp and from : to :443/tcp

  • Filter action: permit

  • Rule: link the list with the action; all interfaces; no tunnel; any authentication method

To script this policy, use the following commands:

ipseccmd -w REG -p "Web traffic packet filter" -r "Block everything" -f *+0 -n BLOCK -xipseccmd -w REG -p "Web traffic packet filter" -r "Permit web traffic" -f *+0:80:TCP -f *+0:443:TCP -n PASS

Note in these examples that a plus sign (+) replaces the equals sign (=) between the source and destination IP address:port:protocol specifications. The + tells the policy agent to build "mirror" rules, which are required for reply traffic to leave the Web server. Without the mirror, you'd need to write separate rules permitting outbound traffic from the server's TCP ports 80 and 443. When you create rules in the GUI, they're mirrored automatically. Figure 1 shows the filter list for Web traffic in the GUI. Figure 2 shows the Web packet filter policy with two rules, one to block all traffic and another to permit Web traffic.

Could you accomplish the same thing with a host-based firewall? Of course, and on a Windows 2003 server that's what I'd prefer. But for a Win2K server, using IPsec to filter packets is very effective at reducing your attack surface area. Think about the roles of various servers in your organization and start to develop IPsec policies that are appropriate for those roles. Use Group Policy to assign the IPsec policies based on organizational units (OUs) that reflect the roles. You can measurably increase the security of your environment simply by implementing a method to limit the traffic that enters a server.

Using IPsec for Domain Isolation
You know who your users are—they have to authenticate to a domain controller (DC) when they want to use network resources. But what about their computers? Sure, some of them are joined to your domain. Windows doesn't require that computers belong to the domain, though. So long as a user possesses valid credentials, he or she can access network resources from any computer on the network. And XP's credential manager makes it even easier for a client that isn't joined to your domain to access network resources.

The concept of domain isolation essentially prohibits rogue machines from accessing domain resources and permits only authorized domain-joined computers to communicate with other authorized computers. You can trust domain-joined computers at least somewhat, because they use security-related technologies that you can centrally control and manage, such as Group Policy, security templates, software restriction settings, IPsec policies, and Microsoft Systems Management Server (SMS). Computers whose configuration you control are computers that do only what you allow them to do. These computers are far less dangerous in your environment than rogue unmanaged machines whose existence or configuration is unknown to you. I urge you to start thinking about domain isolation soon.

Implementing domain isolation across your domain is easier than you might think. First, add the following IPsec policy to your default domain Group Policy Object (GPO):

  • Filter list: Use the existing All IP Traffic sample filter list

  • Filter action: Encapsulating Security Payload (ESP) only, null encryption, SHA-1 integrity; require security; don't communicate with non-IPsec machines

  • Rule: link the list with the action; all interfaces; no tunnel; Kerberos authentication; no default response

Use ESP-null because you're interested in per-packet authentication, not encryption. Privacy isn't a requirement—only authentication and integrity, which SHA-1 hashes provide. You could use IPsec Authentication Header (AH) rather than ESP-null, but then your policy wouldn't work with devices that communicate over Network Address Translation (NAT) devices.

You also need to create a policy that exempts your DCs, because clients need to communicate with them to authenticate and get the Kerberos ticket that's used for all other communications:

  • Filter list: filters with the addresses or address ranges of your DCs

  • Filter action: permit

  • Rule: link the list with the action; all interfaces; no tunnel; any authentication method

You'll need similar policies for machines that can't participate in IPsec, such as network printers.

After you test and deploy these policies, nondomain machines will be unable to communicate with any domain-joined machine because domain members require IPsec. Clients can't get the policy unless they join the domain. Clients outside the domain can't "roll their own" policy because the IPsec policy requires Kerberos, which works only if the client is in the domain. All domain members receive the policy and therefore can communicate with all other domain members.

The network edge is moving inward; indeed, it's becoming more difficult to identify the outside border of a network the way we once did. Instead of an egg with a hard perimeter and a gooey inside, we now view the network as more like an onion with nested layers. We're arriving at a time in which every layer—every computer—must be secured. IPsec is the right technology to help fortify every host in your network.

Project Snapshot: How to

PROBLEM: Create IPsec policies for three purposes: to stop worms from spreading on your network, to protect servers, and to isolate a domain.WHAT YOU NEED: Windows 2003, XP, or Win2K; Netsh Ipsec command for Windows 2003, ipseccmd.exe for XP, or ipsecpol.exe for Win2K.DIFFICULTY: 3 out of 5PROJECT STEPS:




Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.

You May Also Like