Using IPSec to Secure Communications

IP Security (IPSec) is extremely robust. The protocol can create a tunnel to form a VPN (to secure communications between networks or between individual computers and a network), or it can secure communications between individual computers on a per-connection basis. By design, IPSec secures communications between end systems—meaning that routers, gateways, firewalls, and other network infrastructure components between the end systems don't participate in the secure communication and simply pass the packets on to their intended destinations. You can secure communications through encryption, authentication, or both.

Establishing secured communication between end systems is a two-phase process. Phase I involves policy negotiation, key material exchange, and authentication. Phase II involves policy negotiation, session key material refresh or exchange, and establishment.

Phase I's purpose is to establish a security association (SA). After an SA is established, each end system possesses a set of keys that the systems then use to encrypt the Phase II traffic that they exchange. As part of Phase I, each end system must identify and authenticate itself to the other. Windows 2000 supports three authentication methods: Kerberos v5 (the default), certificates issued by a trusted Certificate Authority (CA), and preshared keys. Kerberos v5 works only for Win2K machines that are members of the same domain or of domains with which an established trust relationship exists. To use certificates, you need to have a CA in place that can issue IPSec certificates to computers. (For information about deploying Microsoft Certificate Services, see "Securing Win2K with Certificate Services," September 2001, InstantDoc ID 22113.)

Phase II's purpose is to negotiate SAs and generate the keys that the end systems then use to encrypt the IP traffic sent between them. The systems can reuse valid Phase I SAs to establish multiple Phase II SAs. (Phase I SAs expire and systems must establish new SAs regularly to ensure the integrity of IPSec communications. Phase II SAs expire and are reestablished separately from Phase I SAs.)

End systems attempt to use IPSec to establish secure communications only when IP traffic matches a rule in an IP filter list. You can base these rules on source and destination IP addresses (or the subnets on which those addresses reside), a protocol (such as TCP or UDP), or a source or destination port number, if applicable. The set of rules that governs which traffic IPSec is to secure and which encryption and authentication options IPSec is to use is called an IPSec policy.

For more information about IPSec and how it can secure communications, see Michael Howard, "Defense In-depth," February 2001, InstantDoc ID 16526. Another good source of information is Chapter 8 of the TCP/IP Core Networking Guide volume in the Microsoft Windows 2000 Server Resource Kit.