IPsec

IPsec isn’t straightforward to understand or explain, as it’s not a single protocol. It’s more accurately described as a collection of open standards, protocols and functions designed for interoperability between any two given devices on the Internet.

Overview

Short for Internet Protocol Security, IPsec provides confidentiality, integrity and authentication for communications between networks, and between endpoints when IPv6 is deployed. It was actually developed as a native feature of IPv6, but doesn’t seem in any way ‘mandatory’ as is often claimed. As we’ll see, the only obvious difference between IPv4 and IPv6 in relation to IPsec is Network Address Translation. Forget the articles that drone on about how IPv4 addresses are running out – the real problem is our reliance on NAT.

There are two modes of use for IPsec: In Transport Mode, only the TCP payload – the content of the traffic – is encrypted, while all other components such as headers and routing information are unprotected. The more commonly-used TLS/SSL might have been developed simply to make this form of encryption more convenient and readily implemented. In Tunnel Mode, the entire TCP/IP packet is encrypted and encapsulated within another packet. This outer packet is routed to the destination IPsec tunnel gateway, decrypted and forwarded.

In order to get an abstract understanding of how IPsec works, we only need to look at four central components:

  • Authentication Headers (AH) provide integrity and authentication.
  • Encapsulating Security Payload (ESP) provides confidentiality, authentication and integrity.
  • Security Associtions (SA) provide the algorithms and data that enable the use of AH and ESP operations.
  • Internet Security Association and Key Management Protocol (ISAKMP) for authentication and key exchange.

Authentication Header

The Authentication Header (AH) ensures integrity and authentication of IP packets. It protects the IP payload and all header fields, except for those that might legitimately be modified by the routing systems. It’s mainly used for authenticating the sender of the packet, and for detecting any alterations to the payload as it was routed between the source and destination.

In Transport Mode the payload is hashed, and an integrity check value is placed in an AH header field. In Tunnel Mode, an Integrity Check Value (ICV) is generated for the whole encapsulated packet. At first, it appears anyone could intercept this, alter the payload, rehash it and put the new Integrity Check Value in the header. To eliminate this possibility, both part/ies share a secret key when establishing the connection (I’ll come to this in a minute), and that secret key is hashed alongside the payload to generate the ICV.

Unfortunately IP addresses are also included when generating the ICV, which is what makes this component of IPsec incompatible with Network Address Translation, so this only works between IPsec gateways until IPv6 is fully deployed.

Encapsulating Security Payload

The Encapsulating Security Payload (ESP) can provide authentication, integrity and confidentiality of TCP/IP packets, working in either encryption only or authentication only configuration. In Tunnel Mode, the entire encapsulated packet is protected, and but the authentication method here only covers the payload and ESP headers. It also adds some padding to improve the efficiency of whatever block cipher might be used. Anyone intercepting the traffic won’t be able to determine the payload or its type.

Together, AH and ESP provide the essential components of a VPN connection when used in Tunnel Mode between two gateways. When implemented alongside IPv6, this type of connection could be made between two clients on separate LANs, which is currently impossible with Network Address Translation.

The Security Associations Database and Key Exchange

Security Associations are simply the authentication and encryption algorithms being used for an IPsec connection, plus the keys. They are initialised as the IPsec link is established, using the Internet Association and Key Management Protocol (ISAKMP). This information is maintained in a Security Associations Database (SADB), hich specifies the following:

  • Authentication algorithms
  • Authentication secret
  • Encryption algorithm
  • Secret key
  • Key exchange parameters

A Security Association is one-way, so at least two are required for secure communication between two entities, and the SADB should potentially scale up to support secure multicasting over IPsec.

As with any secure connection, the problem is two endpoints must share a secret key without a third-party intercepting it, and this must happen before any IPsec encryption or authentication is set up. With TLS/SSL, this is acheived using asymmetric encryption to exchange a symmetric encryption key. With IPsec, the common method is the Internet Key Exchange (IKE/ISAKMP), which is normally based on the Diffie-Hellman algorithm.

Basically two communicating parties each generate a random number, which is a secret key used in a mathematical operation to generate two more values – one public and the other private. The two entities exchange their public values, and use them in another mathematical operation that should generate the same result on both sides – this is the shared secret key.