VPN AH & ESP

Authentication Header and Encapsulating Security Payload

Both AH and ESP operate at the network layer of the OSI model and, as a result, have their own protocol numbers for protocol identification carried out by devices in the VPN path. (The protocol numbers assigned are 51 and 50, respectively.) As mentioned earlier, ESP can provide the optional encryption function for data traversing the VPN

connection. Therefore, ESP is the preferred choice for use with IPsec. The data encryption function provided by ESP is carried out by one of the following symmetric key algorithms:

 Digital Encryption Standard (DES)

 Triple DES (3DES)

 Advanced Encryption Standard (AES) (preferred)

The origin authentication, provided by both AH and ESP, can be carried out by one of the following hash algorithms:

 Message digest 5 algorithm (MD5)

 Secure Hash (SHA) (and only for IKEv2: SHA256, SHA384, SHA512)

AH is unavailable for use on the ASA because of the lack of an encryption option. Therefore, when configuring a VPN, only ESP is available to us. Because ESP and AH operate at the network layer, the original host and destination IP addresses remain in the packet throughout the network, exposing them to potential attackers of the VPN connection. However, which IPsec mode (either Transport or Tunnel) is chosen determines the amount of the original packet to be hidden.

Normal Frame:

Ethernet Header

IP Header

TCP/UDP Header

DATA

AH Transport mode frame

Ethernet Header

IP header Authenticated

AH Header Authenticated

TCP/UDP Header Authenticated

DATA Authenticated

AH Tunnel mode frame

Ethernet Header

New IP Header Authenticated

AH Header Authenticated

IP Header Authenticated

TCP/UDP Header Authenticated

DATA Authenticated

                                                                                                                                          

ESP Transport mode frame  

Ethernet Header

IP Header

ESP Header Authenticated

Encrypted TCP/UDP  Header Authenticated

Encrypted Data Authenticated

ESP Trailer Authenticated

ESP Authentication Authenticated

                                                               

ESP Tunnel mode frame                                                        

Ethernet Header

New IP Header

ESP Header Authenticated

Encrypted IP Header Authenticated

Encrypted TCP/UDP Header Authenticated

Encrypted Data Authenticated

ESP Trailer Authenticated

ESP Authentication Authenticated

                                                                              

In both AH and ESP Transport mode, the original IP addresses remain untouched and are visible to potential attackers. However, when operating within Tunnel mode, the AH and ESP headers are placed after the original IP header, and a new IP header is added. This header contains the IP addresses of the VPN endpoints (ASA, PIX, concentrator, or router), which are generally public IP addresses and contain no information, thus not allowing an attacker to determine any valuable information about the internal network. ASA, as a VPN tunnel endpoint, supports only Tunnel mode. Even if Transport mode is configured on the ASA, the resulting VPN tunnel negotiates and uses Tunnel mode. This is also the case for Cisco routers running IOS. However, this restriction applies only to native IPsec functionality, Transport mode is supported on IOS routers (for example, when generic routing encapsulation [GRE] tunneling is used along with IPsec, but not on the ASA, which does not support GRE termination).