Access Denied: Using IPSec with NAT

Q: We'd like to switch from PPTP to Layer Two Tunneling Protocol (L2TP) as the VPN protocol for our traveling users. We've set up everything on the RRAS server and client workstations to support L2TP, including computer certificates to support machine-level authentication, but our client workstations refuse to use L2TP when a traveling user tries to connect from within a client's network. The same user can use L2TP when connected directly to the Internet through a dial-up ISP connection. Is there something special that we need to configure on our clients' firewalls?

Your problem arises because of an incompatibility between IP Security (IPSec) and Network Address Translation (NAT). As you probably know, L2TP uses IPSec for computer-to-computer authentication as well as encryption and integrity checking of VPN traffic. Because IPSec integrity-checks the entire packet, including TCP and UDP headers, IPSec connections don't work over a firewall that performs NAT.

TCP and UDP headers contain a checksum based on information in the packet, including the source and destination IP address. Because a NAT firewall must translate internal client IP addresses to its external Internet IP address and port numbers, a NAT firewall must update the checksum. The IPSec agent at the remote computer detects the change and rejects the packet.

The Internet Engineering Task Force (IETF) is working on IPSec NAT Traversal (NAT-T), a new standard that will update IPSec to be able to traverse NAT boundaries. Windows XP and earlier clients that have the Microsoft L2TP/IPSec VPN Client support the IPSec NAT-T draft, but Windows 2000 RRAS servers don't. Windows Server 2003 RRAS will include support for NAT-T, which should solve your problem. Until then, you'll need to stick with PPTP.