Network Security Fundamentals [Electronic resources] نسخه متنی

IP Security

You cannot talk about VPNs without saying something about IP Security (IPSec). IPSec is a framework of open standards. It is not bound to any specific encryption or authentication algorithm keying technology. IPSec acts on the network layer, where it protects and authenticates IP packets between participating peers such as firewalls, routers, or concentrators. IPSec security provides four major functions:

• Confidentiality
  The sender can encrypt the packets before transmitting them across the network. If such a communication is intercepted, it cannot be read by anybody.

• Data integrity
  The receiver can verify whether the data was changed while traveling the Internet.

• Origin authentication
  The receiver can authenticate the source of the packet.

• Antireplay protection
  The receiver can verify that each packet is unique and is not duplicated.

Encryption

When packets are traveling on the Internet, they are vulnerable to eavesdropping. Clear-text messages can be intercepted and read by anybody. Therefore, to keep the data secure, it can be encrypted. For encryption to work, both the sender and the receiver need to know the rules that were used to encrypt the original message. This is explained in more detail in Chapter 4, "Cryptography." There are two types of encryption:

• Symmetric

• Asymmetric

With symmetric key encryption, each peer uses the same key to encrypt and decrypt data. With asymmetric key encryption, each peer uses a different key to encrypt and decrypt the message. Both the Data Encryption Standard (DES) and Triple DES (3DES) require a symmetric shared secret key. The problem is then to give those keys to both users. The keys can be sent by mail, courier, or public key exchange. The easiest method to exchange the key is Diffie-Hellman public key exchange. This key exchange provides a way for the users to establish a shared secret key, which only they know, although they are sending it over an insecure channel.

Public key cryptosystems rely on a two-key system:

• A public key, which is exchanged between the users

• A private key, which is kept secret by the owners

The Diffie-Hellman public key algorithm states that if user A and user B exchange public keys and combine them with their private keys, the end result should be the same. This is shown in Figure 12-5.

Figure 12-5 is greatly simplified to ensure that the concept of Diffie-Hellman key exchange is clear. There are different variations to this algorithm, know as DH groups 1 through 7. During tunnel setup, VPN peers negotiate which DH group to use.

Encryption can also be accomplished by using the Rivest, Shamir, and Adelman (RSA) algorithm. The RSA algorithm uses an asymmetric key for encryption and decryption. Each user generates two keys: a private key and a public key. The users keep the private key for themselves and exchange the public key. To send an encrypted message to the other end, the local end encrypts the message by using the remote end's public key and the RSA encryption algorithm. This message is then sent to the other end, where it is decrypted using that site's private key. With RSA encryption, the opposite can also be true. The remote end can encrypt a message using its own private key, and the receiver can decrypt the message using the sender's public key. This RSA encryption technique is used for digital signatures and is covered in more detail later in this chapter.

Data Integrity

Data integrity is also a critical function of VPN because data is sent over a public network and can be intercepted and modified. To guard against this interception, every message has an attached hash. This hash guarantees the integrity of the message. The receiver checks this by comparing the received hash with the hash it calculates from the message itself. If both values are equal, the message has not been tampered with. However, if there is no match, the receiver knows that the message was altered.

IPSec uses the Hashed Message Authentication Codes (HMAC) protocol to calculate the hash. At the sender's end, the message and the shared key are sent through a hash algorithm, which produces a hash value. Basically, this hash algorithm is a formula used to convert a variable-length message into a fixed-length hash. It is also important to understand that this is a one-way function. A message can produce a hash, but a hash cannot produce the original message. After the hash is calculated, it is sent over the network together with the message. At the other end, the receiver performs the same action. It sends the message and the shared key through the hash algorithm and then compares the two hashes to verify whether they match.

Two HMAC algorithms are commonly used:

• HMAC-MD5
  This protocol uses a 128-bit shared key. The key and the message are combined to a 128-bit hash.

• HMAC-SHA-1
  This protocol uses a 160-bit shared key. The length of the hash is 160 bits. This protocol is considered stronger because of the longer key.

Origin Authentication

Another important function is origin authentication. Before the electronic era, a seal or a signature on a letter guaranteed its origin. In the electronic era, a document is signed with the sender's private encryption key. This is also called a

digital signature . This signature can be authenticated by decrypting it with the sender's public key. When doing business over a long distance, it is important to know who is at the other side of the phone, fax, and so on. The same is true for VPNs. The devices at the other end of the tunnel must be authenticated before the path is considered secure. There are three peer authentication methods:

• Preshared keys
  A secret key is entered into each peer manually.

• RSA signatures
  The exchange of digital certificates authenticates the peers.

• RSA encryption nonces
  Nonces (a random number generated by the peers) are encrypted and then exchanged between peers. The two nonces are used during the peer authentication process.

Preshared Keys

If preshared keys are used, the same key is configured on each IPSec peer. At each end, the preshared keys are combined with other information (device-specific information) to form the authentication key. They are both sent through a hash algorithm to form a hash. Then the hash is sent to the other site, as you can see in Figure 12-6.

If the remote peer is able to independently create the same hash, the local peer is authenticated. After that, the authentication process continues in the opposite direction. The remote peer combines its specific information with the preshared key and sends the resulting hash to the local peer. If this peer can create the same hash from its stored information and the preshared key, the remote peer is authenticated. Each peer must authenticate its opposite peer before the tunnel is considered secure. This system with preshared keys is easy to configure manually but does not scale very well. Each IPSec peer must be configured with the preshared key of every other peer with which it wants to communicate.

RSA Signatures

With RSA signatures, both hashes are not only authenticated but also digitally signed. Digital certification is discussed in Chapter 13, "Public Key Infrastructure." At the local end, the authentication key and identity information are sent through the hash algorithm to form the hash, a process similar to that used with preshared keys. But with RSA signatures, the hash is then encrypted using the local peer's private key. The result of this procedure is a digital signature, as you can see in Figure 12-7. The digital signature and a digital certificate are both forwarded to the other site. The public encryption key that is also used to decrypt the signature is included in the digital certificate.

At the remote peer, the peer authentication is a two-step process. First, the remote site verifies the digital signature by decrypting it with the public key. The result should be the same hash that the local end made. Next, the remote peer independently creates a hash from its stored information and the authentication key, and this also results in a hash. If the hashes are equal, the local peer is authenticated.

After the local peer is authenticated, the process starts all over in the opposite direction. With this kind of authentication, both peers must authenticate their opposite peer before the tunnel is considered secure.

RSA-Encrypted Nonces

RSA-encrypted nonces require that each site generate a nonce. As stated previously, a nonce is a pseudorandom number. The generated nonces are then encrypted and exchanged. When the other side receives the nonces, it makes an authentication key from both nonces and some other information. That nonce-based key is then combined with device-specific information and run though the hash algorithm, as shown in Figure 12-8. After this, the process is similar to that used for RSA signatures.

NOTE

The word nonce comes from "number used once."

Antireplay Protection

Antireplay protection verifies that each packet is unique and not duplicated. IPSec packets are protected by comparing the sequence number of the received packets and a sliding window on the destination host. Packets in which the sequence number is before the sliding window are considered late, or duplicate. These packets are dropped.

Protocol Framework

The previous sections discussed encryption, integrity, and authentication. Now let's apply these three concepts to the IPSec protocol suite. IPSec is a framework of open standards. IPSec relies on existing technology, such as DES and 3DES, to secure the communication between two entities. There are two main IPSec framework protocols available:

• Authentication header (AH)

• Encapsulating security payload (ESP)

AH

AH is the protocol to use when confidentiality is not required. It provides data authentication and integrity for IP packets between two systems. It verifies that the origin of the packet is correct and that the packet is not modified during transport. It does not encrypt the data packet, so the text is transported in clear text.

Authentication is achieved by using a one-way hash function to create a message digest. The hash is then combined with the text and transmitted to the other site. When the packet reaches its destination, the receiver performs the same one-way hash function and compares the result with the message digest that the sender has supplied. Because the one-way hash uses a symmetric key between the two systems, the authenticity of the packet is guaranteed. The AH function is applied to the entire datagram, except for some header fields that change in transit, such as the Time-To-Live field. The workings of AH are shown in Figure 12-9 and are spelled out in the following steps:

AH supports both HMAC-MD5 and HMAC-SHA-1 algorithms.

ESP

ESP can be used to provide encryption and authentication. It provides confidentiality by performing encryption at the IP packet layer. ESP provides authentication for the IP packet payload and the ESP header. As with AH, ESP verifies the following: that the packet originated from where it declares it did, that it is what it declares it is, and that the packet was not modified during transport.

ESP provides confidentiality by encrypting the payload. It supports several symmetric encryption algorithms. The default for IPSec is 56-bit DES, but Cisco products also support 3DES and AES for stronger encryption. ESP can be used alone or in combination with AH. Between two security gateways, the original data is well protected because the entire IP packet is encrypted. An ESP header and trailer are added to the encrypted payload, as shown in Figure 12-10.

With authentication, the encrypted IP datagram and the ESP header and trailer are included in the hashing process. A new IP header is appended to the front of the packet. This new IP header is used to route the packet through the Internet. When both ESP authentication and encryption are selected, encryption is performed before authentication. One of the main reasons for this order of processing is that it facilitates rapid detection and rejection of incorrect packets at the receiving side. Before decrypting the packet, the receiver can check the authentication of the packets. This requires less processing time and can reduce the impact of denial-of-service (DoS) attacks.

Tunnel or Transport Mode

Both ESP and AH can be applied to IP packets in two different ways:

• Transport mode

• Tunnel mode

These two different modes provide a further level of authentication or encryption support to IPSec. The sections that follow discuss these two IPSec modes in more detail.

Transport Mode

This mode is primarily used for end-to-end connections between hosts or devices acting as hosts. Transport mode protects the payload of the packet but leaves the original IP address readable. This address is used to route a packet through the Internet. Transport mode provides security to the higher layer protocols only. Figure 12-11 shows how transport mode affects AH IPSec connections.

The Layer 3 and Layer 4 headers are pried apart, and the AH is added between them.

Figure 12-12 shows ESP transport mode. Again, the IP header is shifted to the left, and the ESP header is inserted. The ESP trailer and ESP authentication are then appended to the end of the packet.

Although the original header remains intact in both situations, the AH transport does not support Network Address Translation (NAT) because changing the source address in the IP header would cause the authentication to fail. If NAT is needed with AH transport mode, make sure that NAT happens before IPSec. ESP transport mode does not have this problem. The IP header remains outside the authentication and encryption area.

Tunnel Mode

IPSec tunnel mode is used between gateways such as routers, PIX firewalls, or VPN concentrators. Tunnel mode is used when the final destination is not a host but a VPN gateway. In this mode, instead of shifting the original IP header to the left and then inserting the IPSec header, the original header is copied and shifted to the left to form a new IP header. The IPSec header is then placed between the new and the original IP headers. The original datagram is left intact. Figure 12-13 shows AH tunnel mode.

Also in this mode, notice that the IP header is part of the authentication and that is does not support NAT. In Figure 12-14, you can see a depiction of the ESP tunnel mode. The entire original datagram can be encrypted and authenticated. When both are needed, encryption has to be performed first. This allows authentication to be done with assurance that the sender does not alter the datagram before transmission, and the receiver can authenticate the datagram before decrypting the packet. ESP supports NAT in either tunnel or transport mode, and only ESP supports encryption.

Transform Sets

The protocol that brings all the previously mentioned protocols together is the Internet Key Exchange (IKE) protocol. IKE operates in two separate phases when establishing IPSec VPNs.

IKE Phase 1 is responsible for

• Authenticating the IPSec peers

• Negotiating an IKE security association among the peers

• Initiating a secure tunnel for IPSec using the Internet Security Association and Key Management Protocol (ISAKMP)

IKE Phase 2 is responsible for

• Negotiating the set of security parameters for the tunnel

• Creating the IPSec tunnel

Configuring IPSec on a Cisco router is fairly simple. You need to identify some parameters for IKE Phase 1, such as:

• Encryption algorithm
  56-bit DES or the stronger 168-bit 3DES

• Hash algorithm
  MD5 or SHA-1

• Authentication method
  Preshared keys, RSA digital signatures, or RSA encrypted nonces

• Key exchange method
  768-bit Diffie-Hellman group 1 or 1024-bit Diffie-Hellman group 2

• IKE SA lifetime
  86,400 seconds or 1 day

These parameters need to be identical on both sides, or the connection will not be established. Once these are configured, the only other values you need to supply to establish the IPSec tunnel in IKE Phase 2 mode are as follows:

• IPSec protocol
  AH and/or ESP

• Hash algorithm
  MD5 or SHA-1

• Encryption algorithm for ESP
  DES or 3DES

NOTE

Besides DES and 3DES, a new encryption algorithm is now available, Advanced Encryption Standard (AES). AES comes in three varieties:

• AES
  Specifies 128-bit AES as the encryption algorithm

• AES192
  Specifies 192-bit AES as the encryption algorithm

• AES256
  Specifies 256-bit AES as the encryption algorithm

To make the configuration process easier, the IPSec parameters are already grouped into some predefined configurations called

transform sets . The transform sets identify the IPSec protocol, hash algorithm, and when needed, the encryption algorithm. The following transform sets are available, as shown in Table 12-1.