Using Authenticated IPsec to Bypass Windows Firewall

A challenge that Windows administrators face when they consider deploying a host-based firewall is how to manage the assets behind the firewall. Although you could open ports that management agents use and specify the hosts that can send traffic to those ports, doing so could be difficult, especially if the agents use dynamic ports or remote procedure calls (RPCs). Recognizing this challenge, Microsoft included a feature in Windows Firewall that lets you configure the firewall to allow authenticated IPsec traffic to pass through it without inspection. To establish an authenticated IPsec session between two hosts, each system must be able to authenticate by using the Microsoft-developed Kerberos extension to IPsec or by using X.509v3 certificates issued for the purpose. Although you can use shared keys to establish IPsec communications, you can't use them to guarantee the identity of hosts; therefore, authenticated IPsec bypass doesn't support this authentication method. The firewall administrator can choose which hosts to trust and to permit IPsec traffic from them through the firewall. You can use these IPsec connections to facilitate management traffic through Windows Firewall, thereby allowing the assets to be managed.

You need to use Group Policy Objects (GPOs) to configure Windows Firewall to permit authenticated IPsec to pass through without inspection. You can't configure authenticated IPsec bypass if you don’t have Active Directory (AD) or your hosts don't have computer accounts in a domain. You can find the authenticated IPsec bypass settings under the Windows Firewall GPO settings in the Microsoft Management Console (MMC) Group Policy Object snap-in. Web Figure 2 shows the dialog box where you configure which hosts may bypass the firewall. The format of the rule follows the Security Descriptor Definition Language (SDDL) and is

O:G:D:()[…]

Although you can specify individual hosts, I recommend that you instead create groups in AD that contain the computer accounts for domain controllers (DCs), management stations, and any other hosts that might need to communicate with systems protected by Windows Firewall. You can then specify the groups in the rule instead of each host. You don’t need to specify Owner and Group SIDs in the rule, but you do need to know the SIDs of the hosts or groups you create and want to configure authenticated IPsec bypass for. You can use the GetSID (getsid.exe) tool in the Windows resource kits or Support Tools to find the SIDs for hosts and groups. For example, for a single group called Management Stations, the format of the rule is

O:DAG:DAD:(A;;RCGW;;;1-5-21-2867579479-937772154-2885998344-1114)

You can specify additional hosts and groups by appending (A;;RCGW;;;) entries to the rule.

After you've configured Windows Firewall, you need to configure the hosts to use IPsec for communications between them. It isn't necessary to encrypt traffic between hosts with Cisco Encapsulating Security Payload (ESP) because integrity using the Authentication Header (AH) protocol is sufficient. For more information about configuring IPsec, see the Windows IT Pro article "IPsec for Network Protection," May 2005, InstantDoc ID 45903, and "Access Denied: Using IP Security Policies to Restrict Access to a Server," March 2005, InstantDoc ID 45217.