Internet Protocol Security (IPSec)

What is IPSec (Internet Protocol Security)?

A protocol for negotiating and controlling the security of transmissions over a TCP/IP internetwork. Internet Protocol Security (IPSec) defines standards for data encryption and data integrity at the level of Internet Protocol (IP) datagrams and can be used to encrypt transmission of data and ensure that the data originated from the sender and was not modified in transit. IPSec encrypts data at the IP level and uses tunneling to securely send information over the Internet and between intranets. IPSec is an emerging Internet Engineering Task Force (IETF) standard and is implemented in the Microsoft Windows 2000 operating system.

How it works

IPSec is implemented at the transport layer of the Open Systems Interconnection (OSI) reference model and protects IP and higher protocols using security policies that can be configured to meet the needs of securing users, sites, applications, or the enterprise in general.

IPSec essentially resides as an additional layer under the TCP/IP protocol stack and is controlled by security policies installed on each machine and by an encryption scheme negotiated between the sender and the receiver. These security policies consist of a collection of filters with associated behaviors. When the IP address, port number, and protocol of an IP packet match a particular filter, the corresponding behavior is applied to the packet.

In Windows 2000, these security policies are created and assigned at the domain level or for individual hosts using the IPSec Management snap-in for the Microsoft Management Console (MMC).

IPSec policies consist of rules that specify the security requirements for different forms of communication. These rules are used to initiate and control secure communication based on the nature of the IP traffic, the source of the traffic, and its destination. These rules specify authentication and negotiation methods, tunneling attributes, and connection types.

To establish a security association (secure communication session) between two computers, a protocol framework called ISAKMP/Oakley is used. ISAKMP/Oakley includes a set of cryptographic algorithms but is also extensible to support user-defined encryption algorithms. During the negotiation process, agreement is reached on the authentication and security methods to be used, and a shared key is generated for data encryption. IPSec supports two different kinds of security associations:

• Authentication Header (AH) protocol:
  Provides user authentication and protection from replay attacks and supports data authentication and integrity functions. AH enables the recipient to be sure of the identity of the sender and that the data has not been modified during transmission. AH does not provide any encryption of the data itself. AH information is embedded in the IP packet’s header and can be used alone or with the Encapsulating Security Payload (ESP) protocol.

   

• Encapsulating Security Payload (ESP) protocol:
  Encapsulates and encrypts user data to provide full data confidentiality. ESP also includes optional authentication and protections from replay attacks and can be used either by itself or with AH. ESP information is also embedded in the IP packet’s header.

   

Devices and software configured to support IPSec can use either public key encryption using keys supplied by certificate authorities (CAs) or preshared keys for private encryption.