PPTP vs. L2TP

Which VPN protocol is right for you?

Creating a VPN server with Windows NT 4.0 is simple, but your choices are limited. The OS offers few administrative options, and PPTP is the only VPN protocol available. Windows 2000 brings improved reliability, scalability, and manageability—and support for two VPN protocols. In addition to PPTP, Win2K supports the Layer 2 Tunneling Protocol (L2TP). When choosing which protocol to use to set up your server, you should understand how both protocols work and consider their authentication and encryption features. L2TP offers many improvements over PPTP, but you must understand the enhanced functionality to reap its benefits. Becoming familiar with PPTP and L2TP will help you pick the best VPN configuration for your situation. (For more information about Win2K VPNs, see "Related Articles in Previous Issues," page 26.)

A VPN Primer
A VPN is a secured tunnel from a remote user's computer, through the Internet, directly to your organization's private network—a pipe on top of the existing public network. A VPN gives users secure access to a private network over almost any type of Internet connection. In Microsoft's VPN implementation, the only thing needed between the client computer and the VPN server is an IP-based network. If the VPN client has an Internet connection and the VPN server has the same, you're all set.

VPNs have saved companies large amounts of money. Instead of paying for costly point-to-point connections, such as T1, ISDN, or frame-relay connections, a company can use its existing public Internet connection. VPNs let any site around the world create secure tunnels to any other site, with little administrative effort. VPNs have revolutionized the way companies communicate, and Microsoft has provided the tools in Win2K to make VPNs flexible and easy to set up.

The PPP Foundation
How different are PPTP and L2TP? If you were to compare and decode the data from both protocols in the Open System Interconnection (OSI) reference model, you would find one primary similarity: their reliance on the Point-to-Point Protocol. PPP is the foundation for both VPN protocols and is the protocol that encapsulates the data you transfer (i.e., the payload) over a private network. PPTP and L2TP then add another layer of encapsulation to tunnel the payload through a public network.

Table 1 shows the layers in which certain protocols operate within the OSI model. PPP, in the data-link layer of the OSI model, was originally developed to encapsulate data and carry it over point-to-point links. If your company has any type of point-to-point connection, such as a T1 line, your router probably uses PPP encapsulation. You can also use this protocol for asynchronous (i.e., dial-up) connections. Your remote users' Win2K or Windows 9x dial-up settings probably show that their systems dial in to a PPP server.

PPP provides many benefits—authentication and compression, for example—that its older cousin, Serial Line Internet Protocol (SLIP), doesn't provide. A subset of PPP protocols handles connections' operations: The PPP Link Control Protocol (LCP) establishes, configures, maintains, and terminates a point-to-point connection, and the PPP Network Control Protocol (NCP) establishes and configures different network-layer protocols over the PPP link. You can run Novell IPX and Microsoft IP simultaneously over one PPP link.

PPP is obviously an important part of PPTP and L2TP. PPP lets you use PPTP or L2TP to run remote applications that depend on nonroutable protocols. PPTP and L2TP are identical at the physical and data-link layers, but their similarities end there.

Tunneling
Tunneling protocols, such as PPTP and L2TP, encapsulate and usually encrypt data for transfer from one point to another over a public network. But before the tunneling encapsulation takes place, the PPP encapsulation occurs.

In the PPP encapsulation, one protocol data unit (PDU) is placed inside another PDU on the first PDU's way down the OSI model. For example, TCP (in the transport layer) is encapsulated by IP (in the network layer), which in turn is encapsulated by PPP (in the data-link layer).

Tunneling protocols are higher-layer protocols that transport encapsulated payloads. The VPN protocol encapsulates the already-encapsulated payload and sends the payload between the endpoints of the tunnel. After the far endpoint of the tunnel receives the payload, it decapsulates and processes the payload.

Good Ol' PPTP
The PPTP protocol encapsulates IP packets for transmission over an IP-based network. PPTP clients use the destination TCP port 1723 to create a PPTP control connection for a tunnel. This process takes place in the transport layer. After the host and client establish a tunnel, they send PPTP control-connection packets back and forth to maintain the connection. These control-connection packets consist of PPTP Echo-Request and PPTP Echo-Reply messages.

In addition to the PPTP control connection, which the protocol uses for tunnel maintenance, PPTP uses a connection for data tunneling. Data encapsulation works differently in a tunneling scenario than it does in a typical data transfer (e.g., Telnet data transfer). Tunneled data actually goes through two encapsulation steps. To create the PPP payload, data travels down the OSI model starting at the application layer and ending at the data-link layer. After the payload creation, the data moves back up the OSI model and higher-layer protocols encapsulate the payload.

When the data reaches the transport layer, the protocol can't send the payload because that task is the data-link layer's responsibility. PPTP handles the Layer 2 responsibilities that usually belong to PPP and adds a PPP header and trailer to the PPTP data structure. PPTP encrypts the payload, then encapsulates it with a PPP header to create a frame in the data-link layer. PPTP then encapsulates the PPP frame in a Generic Routing Encapsulation (GRE) packet, which operates in the network layer. GRE provides a way to encapsulate Layer 3 protocols, such as IPX, AppleTalk, and DECnet, for IP networks, but it lacks the functionality to set up a session or provide security. Thus, you use a PPTP control connection to provide the session setup and security. Using GRE as the encapsulation method restricts the use of PPTP to IP-based networks.

After encapsulating the PPP frame with a GRE header, PPTP encapsulates the frame with an IP header. This IP header contains the source and destination addresses for the packet. Finally, PPTP adds a PPP header and trailer. Figure 1 shows the data structure of PPTP tunneled data.

The source system then sends the data through the tunnel. The target system strips the data of all its headers and trailers until the system reaches the PPP payload.

The New Kid on the Block
L2TP is a combination of PPTP and the Layer 2 Forwarding (L2F) protocol. PPTP tunnels PPP, and L2F tunnels SLIP and PPP. After Cisco Systems designed L2F, the Internet Engineering Task Force (IETF) directed the company to combine PPTP and L2F into one protocol to avoid confusion and interoperability problems in the marketplace. L2TP supposedly has the best features of PPTP and L2F.

A major enhancement in L2TP is that it runs over non-IP-based networks, including asynchronous transfer mode (ATM), X.25, and frame-relay networks. However, you don't get this benefit in a Win2K implementation because the OS supports only IP.

L2TP uses the same message format for tunnel maintenance as it does for data tunneling. UDP is the transport-layer protocol of choice for L2TP. In Microsoft's L2TP implementation, control messages are encrypted PPP payloads sent over IP as UDP messages, and sequencing ensures data delivery. L2TP messages include a Next-Received field and a Next-Sent field, which are comparable to TCP's Acknowledgement Number field and Sequence Number field, respectively.

As with PPTP, L2TP data tunneling begins with a PPP payload. L2TP encapsulates the PPP payload with a PPP header and an L2TP header, which results in an L2TP-encapsulated packet. UDP then encapsulates this packet. L2TP uses UDP port 1701 for both the source and destination port. Depending on the type of IP Security (IPSec) policy you've selected, L2TP might encrypt the UDP message and add an IPSec Encapsulating Security Payload (ESP) header and trailer as well as an IPSec Authentication trailer. L2TP then encapsulates this IPSec packet with an IP header, which contains the source and destination addresses. Finally, L2TP performs a second PPP encapsulation to prepare the data for transmission. Figure 2 shows the data structure of L2TP tunneled data.

When the destination computer receives the data, it processes the PPP header and trailer, then removes the IP header. The computer uses the IPSec Authentication trailer to authenticate the IP payload, then uses the IPSec ESP header to decrypt the packet.

Next, the computer processes the UDP header, then uses the L2TP header to identify the L2TP tunnel. The PPP payload is the only data left, and the server either processes this data or forwards it to the proper destination.

Security
Security is the most important feature of a VPN. All the data coming from clients is traveling through the Internet to the VPN server. This VPN server could be 20 hops away, traveling through five ISPs, and the client's data is handled quite a bit on the way to your company's network. How can you be sure that the data makes it to your VPN server without anyone viewing it? You can employ authentication and encryption.

To provide user authentication, PPTP uses one of several PPP-based authentication protocols, including Extensible Authentication Protocol (EAP), Microsoft Challenge Handshake Authentication Protocol (MSCHAP) version 1 and version 2, Challenge Handshake Authentication Protocol (CHAP), Shiva Password Authentication Protocol (SPAP), and Password Authentication Protocol (PAP). MSCHAP version 2 and EAP-Transport Layer Security (TLS) are the most secure protocols because they provide mutual authentication, in which both the VPN server and client verify their partner computer's identity. If a client authenticates through any of the other protocols, the server verifies the client's identity, but the client doesn't verify the server's identity.

PPTP encryption ensures that no one can view the data as it travels over the Internet. Microsoft Point-to-Point Encryption (MPPE) negotiates encryption over a PPTP connection and can be used only with MSCHAP (version 1 and version 2) and EAP-TLS. You can employ one of three encryption key strengths with MPPE: 40-bit, 56-bit, or 128-bit. However, if you support a mixed environment of Windows clients, you must use 40-bit keys because older clients don't support longer keys.

PPTP changes the encryption keys with every received packet. MPPE was designed for point-to-point links in which each data packet arrives sequentially and in which few data packets are lost. In such an environment, the encryption key of one packet could depend on the decryption of the previous packet. In a VPN environment, this setup doesn't work because data packets frequently arrive out of sequence. Thus, PPTP decrypts packets independent of other packets and uses a sequence number to alter the encryption keys so that the decryption process can work without information from the previous packet.

Although PPTP is reasonably secure, it isn't as secure as L2TP over IPSec. L2TP over IPSec provides user- and computer-level authentication as well as data authentication and encryption.

L2TP over IPSec first uses local computer certificates, which you obtain from a certificate authority (CA), to authenticate both VPN clients and servers. The client and server exchange their certificates to create an IPSec ESP security association (SA).

After L2TP over IPSec completes the computer authentication process, it performs user-level authentication. You can choose any PPP-based authentication protocol—even PAP, which sends the username and password in clear text—and the process is still secure because L2TP over IPSec encrypts the session. However, you can make user authentication more secure by using MSCHAP, which uses encryption keys separate from the computer-level encryption.

Because L2TP over IPSec uses the Triple Data Encryption Standard (3DES) algorithm, its data encryption is much stronger than PPTP's. 3DES is for use only in North America and is designed for high-security environments. If you don't need this level of security (and its associated overhead), you can employ DES, which uses one 56-bit key (3DES uses three 56-bit keys).

L2TP over IPSec not only provides computer-level and user-level authentication and data encryption but also offers data authentication. To accomplish data authentication, L2TP over IPSec uses Hash Message Authentication Code (HMAC) Message Digest 5 (MD5). This hashing algorithm creates a 128-bit hash to authenticate data.

A Simple Choice
PPTP and L2TP offer different functionality. L2TP's design lets you use it over non-IP-based networks, and the protocol establishes tunnel maintenance and control using the same message format and protocols. In contrast, PPTP works only over IP and uses a separate TCP control connection for tunnel maintenance. Although PPTP is secure, L2TP over IPSec provides multiple layers of security, which, if used correctly, can almost guarantee that sensitive company data won't be compromised.

These enhancements make L2TP the VPN protocol of the future. As more IT professionals learn about IPSec and its benefits, L2TP over IPSec will become the VPN protocol of choice. Microsoft has made L2TP setup as easy as a few mouse clicks. So give L2TP a try and get the most from your Win2K license.