Definitive MPLS Network Designs [Electronic resources] نسخه متنی

Layer 3 MPLS VPN Service: Design Overview

TK's drivers for introducing a network-based Layer 3 VPN service were similar to those for USCom, as discussed in the preceding chapter. However, the Layer 3 MPLS VPN technology found traction earlier in Kingland. Therefore, TK's network services portfolio is a little more mature, with more VPN sites and several additional services already deployed.

Many drivers forced early adoption of this technology in Kingland as well as other countries in this region. At the time (circa 1998), many new service providers were emerging, challenging the incumbent providers. One way for these new operators to differentiate themselves from the incumbents was to provide additional services above and beyond the traditional Frame Relay/ATM services. TK quickly followed suit and introduced an MPLS-based VPN service that it marketed aggressively. A further reason for the accelerated growth of this technology was that Kingland had a long history of providing managed services (for example, Frame Relay services were managed more often than not). Therefore, it was easier for TK to use Layer 3 MPLS VPN technology to interconnect the CE routers. The historic predominance of managed VPN services also explains why today the vast majority of Layer 3 MPLS VPN customers in Kingland continue to have a managed service.

Because of the maturity of the TK Layer 3 MPLS VPN service, a wide range of customer profiles are targeted, including small and medium businesses (SMBs), teleworkers, Enterprises (small through large), ISPs, and so forth. The design selected for this service is similar in many ways to the one selected by USCom. For example, all the existing POPs are used for Layer 3 MPLS VPN services and offer a wide range of access speeds. The edge of the network is a little different, however, because TK chose to run both Layer 3 MPLS VPN and Internet services from its mPE routers.

TK currently has 450 mPE routers deployed. TK services a total of 2400 VPN domestic clients, with a combined total of 100,000 VPNv4 routes (including remote-access clients). In addition to these customers, the international network has 360 VPN customers. The domestic network offers a Carrier's Carrier service (described later in this chapter) that has five customers, increasing the number of VPNv4 routes carried in the TK backbone to 120,000 (about 18,000 for the international network and about 2000 for the Carrier's Carrier service).

Note

Interprovider connectivity is described in the next chapter. Therefore, the TK international network is not described in this chapter.

TK has defined four different types of service offerings that are available to both domestic and international customers:

Intranet VPN This customer requires connectivity between internal sites for the creation of an intranet. This customer may access TK-owned central services.

Extranet VPN This customer requires connectivity between internal and external partner sites for the creation of an extranet.

Intranet + Internet This is an intranet customer that also requires Internet connectivity. Note that Internet service is bundled along with Layer 3 MPLS VPN as a service package.

Remote-access VPN This is an intranet customer connected via dial, ISDN, or DSL. Note that this service is bundled with the intranet Layer 3 MPLS VPN service.

Moreover, several VPN categories are defined. They help identify the size and scope of the attached customer. These categories are based on collation of statistics gathered by TK over a number of years, as shown in Table 4-1.

Multiservice PE Router Basic Engineering Guidelines

The engineering guidelines governing the addition of VPN clients to the Layer 3 MPLS VPN service are similar to those used by USCom in the preceding chapter. However, because the edge routers are shared between different services, additions to these basic guidelines are necessary so that scale and security are maintained. Chapter 5 contains the details of all the necessary scale and security features for overlapping services.

All services are provisioned from a centralized management system. A number of default templates, such as the ones discussed in the preceding chapter, are used.

TK allocates a /32 IP address to each customer CE router that it manages so that it can access and monitor this device from its Network Operations Center (NOC). This address is taken from a TK allocated address block. It is exported from the customer VRF using a unique route-target value that is used solely by the management VPN. The PE-CE links are addressed from a different TK registered public address block and are not redistributed into an attached customer's IGP. This allows TK to use the same IP address block on every PE-CE link, thus saving TK thousands of addresses.

Customer VRF Naming Convention

Unlike the USCom design, TK decided not to use a representation of a customer name for the VRF name. Instead, it chose to use the real customer name (noting that the maximum length of the name in Cisco IOS is 32 characters). The same name is used for a given customer on all PE routers that attach sites of that customer. TK preferred this approach because it felt this was more intuitive for the operations staff, because they did not need to cross-reference a representative name to a real customer name. This made the name easier to use while troubleshooting connectivity problems.

RT/RD Allocation Schemes

TK chose to use its autonomous system number (32764) within the route distinguisher/route target format. It also chose to use a unique RD per VRF and per PE router (as described in the preceding chapter). This approach has a cost in terms of memory consumption but is required for the design because TK has multiple load-balancing requirements (as you will see in the "Load-Balancing Support" section).

The route-target range 32764:[1-100] is reserved for special use, such as network management and future (as yet unknown) service requirements. The remaining route-target range is available for customer allocation. By default, one unique RT per VPN is allocated to each customer.

Network Management VPN

A management VPN is used to connect to and manage the CE routers. The management VPN imports all the relevant CE router /32 addresses that are allocated by TK. To achieve this, TK evaluated several different options:

Option 1 The PE router that connects to the LAN where all the network management equipment is located imports routes that carry the route-target values that correspond to the export route-target values used for a given customer VPN. The mPE router attaching to the remote customer sites imports routes that carry the route-target value that corresponds to the export route-target value configured in the management VRF.

Option 2 The PE router that connects to the management LAN is configured with a single import/export route-target pair that each customer VRF also has configured in addition to its customer-specific route-target values.

Option 3 The CE router/32 loopback interface addresses are advertised using a unique route-target value that only the management VPN imports.

TK chose to use Option 3 because it had the least overhead in terms of provisioning and also provided much better isolation of the network management system (NMS) because only the CE router addresses are known to the NMS. The NMS reachability information is not redistributed from the CE router into the customer IGP, and customer routes are not known by the NMS. The template shown in Example 4-1 is used for each VRF.

Example 4-1. PE Router Configuration Template for Network Management VPN

ip vrf vrf-name
rd 32764:customer-and-PE-router-specific-value
export map Network_Management
import route-target 32764:10
import route-target 32764:customer-specific-value
export route-target 32764:customer-specific-value
!
access-list 10 permit customer-CE-address
route-map Network_Management permit 10
match ip address 10
set extcommunity rt 32764:11 additive

This configuration template causes the CE router addresses to be advertised with the customer route-target value (so that the CE router addresses are accessible from within the VPN for troubleshooting purposes), as well as the network management route-target 32764:11. The network management route target value is taken from the reserved range that TK uses to import into its network management VPN. An additional import statement covering route-target value 32764:10 is also added so that the network management address can be imported into the VRF for two-way connectivity. This address is not redistributed into the customer IGP at the CE router.

Load-Balancing Support

TK uses a different RD value for each VRF provisioned at the mPE routers. Because TK uses VPNv4 RRs, the use of a unique RD is necessary to support its load-balancing capabilities. The reason for this is that the combination of an RD and IPv4 address constitutes a VPNv4 address, and MP-BGP considers matching addresses comparable within the BGP best path calculation. For example, a VPNv4 prefix of 32764:101:10.1.1.1 originated from an mPE router in the Center-West Level 1 POP would be comparable to a VPNv4 prefix of 32764:101:10.1.1.1 originated from an mPE router in the South-East Level 1 POP. An RR in the Center Level 1 POP therefore receives the same prefix twice but chooses a best path based on the normal BGP selection process and readvertises only that best path to other mPE routers. The result of this is the loss of a given path to a receiving mPE router in another POP, which would prevent it from performing any sort of load balancing. This is illustrated in Figure 4-7.

Figure 4-7 shows that a customer VPN network is dual-attached to mPE routers South-East-PE1 and Center-West-PE1. It advertises reachability for 10.1.1.1/32 to both mPE routers with the same metric. The mPE router Center-PE1 only receives the path via South-East-PE1 and therefore cannot load-balance traffic using the Center-West-PE1 path. To rectify this situation, a different RD may be used at each mPE router (which is the solution chosen by TK). Therefore, as shown in Figure 4-8, both paths can be received by the Center-PE1 mPE router.

Getting the different routes to the mPE routers through the RRs is only part of the problem. To successfully provide load balancing of traffic toward the 10.1.1.1/32 destination over both available paths via mPE routers South-East-PE1 and Center-West-PE1, TK chose to use iBGP multipath functionality to ensure that the mPE routers use multiple routes.

iBGP Multipath Support for VPNv4

The normal operation of MP-BGP is to install a single best path into the routing table. However, this behavior can be changed using the maximum-paths number-of-paths command, which allows up to 16 different paths. MP-BGP uses the best-path algorithm to select one of the available paths as the best path; this path is inserted into the routing table. However, because of the maximum-paths configuration, the other additional paths may also be inserted into the routing table. All these entries can then be used to provide unequal-cost load balancing on a per-source/destination pair basis.

TK limits the number of maximum paths based on a given customer requirement, which is typically only two different paths. The reason is that each extra path requires additional memory and processing at the mPE router, so tight controls on the deployment of this functionality are necessary. TK charges extra for load balancing support because it is considered an additional service above and beyond the basic connectivity to an MPLS-based VPN.

eiBGP Multipath Support for VPNv4

TK also provides an eiBGP multipath service. This support is similar in functionality to iBGP multipath. It is deployed in the TK network for multihomed sites that require load balancing of traffic from the core. The key difference with this feature is that it can take into consideration both internal MP-BGP and external BGP paths from within a VRF for load-balancing purposes. The maximum-paths eibgp number-of-paths command is used to enable this feature on a per-customer basis.

mPE Router Control-Plane Requirements

TK follows the same set of control-plane recommendations as specified in the preceding chapter. This includes the use of path MTU discovery (see [PMTU]), input queue tuning, and Selective Packet Discard (SPD) tuning. In addition, TK also selected a centralized route reflection design with a single level of hierarchy. However, as stated earlier, the IPv4 and VPNv4 reflection is kept separate.

VPNv4 Route Reflector Placement

TK uses its six Level 1 POPs to house its VPNv4 RRs. Each Level 1 POP has two RRs, and a limit of 300 MP-BGP peering sessions is imposed on each reflector. This limit is based on internal TK testing. It does not represent the theoretical maximum number of peering sessions that the RRs could handle.

Each mPE router in a Level 1 POP has redundant MP-BGP peering sessions with the two RRs located in the local Level 1 POP. mPE routers in Level 2 POPs have redundant RR connections to two separate Level 1 POPs. Figure 4-9 shows the RR connectivity in a Level 1 POP.

A full mesh of MP-BGP peering is deployed between all the RRs, as shown in Figure 4-10.

PE-CE Routing Protocol Design

TK offers a service similar to the one discussed in the preceding chapter in terms of static routing and BGP-4 on the PE-CE links. However, because TK manages the CE routers, it is more able to use static routing for the large number of single-attached customer sites, whose sole routing requirement is to advertise their local LAN subnet. Typically a default route pointing toward the mPE router is configured at the CE router, and the LAN subnet is configured as a static route in the VRF at the mPE router. If a site requires more than a small set of subnet advertisements, or if it learns a default route locally via another service provider, or if it needs load-balancing support, BGP-4 is the protocol of choice.