Definitive MPLS Network Designs [Electronic resources] نسخه متنی

Quality of Service Design

The EuroBank network carries a multitude of applications with very different quality of service (QoS) requirements and very different importance to EuroBank's business. At one end of the spectrum, the network will be carrying traffic from a user casually browsing the Internet. At the other end of the spectrum, the network will be carrying brokers' instructions with extremely tight latency constraints as well as the whole company's telephony traffic and its associated tight delay and jitter constraints.

Another interesting challenge is that the EuroBank network is made up of a mix of routinely congested links, occasionally and moderately congested links, and completely uncongested links. This stems from the great variations in bandwidth costs depending on the link span and location, which forces EuroBank to apply different approaches in different parts of the network. For example, the high cost of international bandwidth resulted in EuroBank provisioning bandwidth very carefully on the international POP-POP routes (in the 1040 Mbps range) and having to deal with periods of congestion on those. On the other hand, reasonable costs of Fast Ethernet and STM-1 links in a metro area or sometimes even on long distance within a country allow EuroBank to provision bandwidth more generously on the corresponding POP-POP and POP-office links and having to cope with only occasional moderate congestion on those. Finally, attractively priced metro Gigabit Ethernet services allow EuroBank to overprovision capacity among the London POPs and to never expect congestion on these links.

Another important consideration was that EuroBank did not have experience in how some applications behaved and needed to be handled in the network from a QoS perspective. Hence, it wanted to have some flexibility to nimbly adjust the QoS treatment given to some applications, as proven necessary in practice. As a result, EuroBank adopted the following approach:

It defined service classes that group applications with similar QoS requirements as well as similar levels of importance with respect to EuroBank's business mission. Those are defined independent of the bandwidth management policies (and their associated queues), which actually get deployed in different parts of the network. EuroBank defined nine such service classes.

There is no QoS policy on Gigabit Ethernet-based POP-POP and POP-data center links because those links are overprovisioned.

It adopted a mid-grain three-queue QoS policy on the 100+ Mbps POP-POP, POP-office, and POP-data center links.

It adopted a fine-grain six-queue QoS policy on the sub-100 Mbps POP-POP and POP-office links.

It used the three classes of service available from the managed Layer 3 MPLS VPN and managed Frame Relay service providers, which provide connectivity for the branches.

It mapped the nine service classes into the various QoS policies. Specifically, a mapping is defined for those nine service classes onto the six queues of the fine-grain QoS policy. Another mapping is defined for mapping the service classes onto the three queues of the mid-grain QoS policy. Finally, a mapping is defined to the three classes of service for the managed router services.

This QoS approach applies finer-grain mechanisms in the core than on the edge because links tend to be higher-speed in the edge than in the core, which is thus the primary bottleneck. This is quite characteristic of Enterprise MPLS networks with large geographic coverage (particularly international). This is the opposite of the typical situation in service provider MPLS networks (such as the one for USCom and Telecom Kingland, described in Chapters 3 and 4, respectively). In that case, links tend to be much faster in the core than on the access. Hence, finer-grain QoS mechanisms are deployed on the access, which constitutes the primary bottleneck, rather than in the core.

Recall the N/M/P QoS model defined in the "QoS Models" section of Chapter 2, "Technology Primer: Quality of Service, Traffic Engineering, and Network Recovery." In that model, N is the number of queues on the access, M is the number of queues in the core, and P is set to 0 if MPLS Traffic Engineering is not used. The EuroBank MPLS network adheres to an N/6/0 model. In this model, N=1 for some access scenarios such as Gigabit Ethernet links to data centers, N=3 for some access scenarios such as branches or Fast Ethernet links to offices, and N=6 for some access scenarios such as 834 Mbps leased-line access to offices. In comparison, where its "PE-to-CE" QoS option is used, USCom follows a 3/1/0 model.

EuroBank's Service Classes

Table 6-4 lists the service classes defined by EuroBank. It also lists application examples, the DSCP value used to identify each service class, and the EXP value used in the MPLS core.

EuroBank picked DSCP values such that they automatically map to the desired EXP values using the default DSCP-to-EXP mapping of the IOS PE router implementation. (This copies the 3-bit precedence field into the EXP field of the imposed MPLS label stack entries.) However, Table 6-4 shows that DSCPs from two different service classes are mapped to the same EXP value. Both DSCP=2 of Intranet Batch and DSCP=0 of Default map to EXP=0. This is because EuroBank decided not to use the EXP=7 value and leave it as reserved for future use. Being left with seven EXP values while eight service classes are transported over MPLS (Multicast traffic is not encapsulated in MPLS), EuroBank decided to allocate the same EXP value to the Intranet Batch and Default service classes. This was decided because these two service classes are always grouped in the same queue in all the various QoS policies deployed by EuroBank and thus don't need to be distinguished inside the MPLS core. Note that if in the future these two service classes need to be scheduled in different queues, EuroBank can easily do so by activating the necessary DSCP-to-EXP mapping on PE routers. For example, suppose the Intranet Batch service classes need better treatment and need to be grouped with the Mission-Critical Batch service class. The PE router could be configured to make sure that the EXP value of all imposed label stack entries on packets arriving with DSCP=2 were marked with EXP=2 (without rewriting the IP header's DSCP value). The necessary configuration is shown in Example 6-3.

Example 6-3. PE Router Input Policy for DSCP-to-EXP Mapping

!
class-map match-any Intranet-Batch
match dscp 2
!
policy-map PE-input-DSCP-to-EXP
class Intranet-Batch
set mpls exp imposition 2
...
interface pos0/0
service-policy input PE-input-DSCP-to-EXP

Traffic Classification in Offices and Data Centers

[NBAR]) as well as some of its application-specific extensions to perform the deep packet inspection necessary to support classification of its service classes.

EuroBank uses the CITRIX Independent Computing Architecture (ICA) protocol to support banking transactions with servers located in data centers. As shown in Table 6-4, some CITRIX flows need to be classified as mission-critical interactive, and others need to be classified as mission-critical batch. Example 6-4 shows the configuration EuroBank uses to classify differently the CITRIX ICA background transactions from the other CITRIX transactions using NBAR. It also shows EuroBank's ability to interpret the ICA tag, which reflects the priority of the CITRIX ICA transaction.

Example 6-4. Classification of CITRIX ICA Transactions Using NBAR

!
class-map match-any Mission-Critical-Interactive
match protocol citrix ica-tag "0"
match protocol citrix ica-tag "1"
match protocol citrix ica-tag "2"
match protocol http url "/transact/*"
match ...
match ...
class-map Mission-Critical-Batch
match protocol citrix ica-tag "3"

[DLSw]), which places all SNA traffic inside TCP packets. Multiple methods exist to classify different types of SNA traffic and mark it accordingly. EuroBank elected the method that involves establishing multiple TCP connections for the DLSw+ traffic, each using a well-known port number and corresponding to a required level of priority. It also involves steering different SNA flows (for example, depending on the SAP or MAC address) into the appropriate TCP connection. Then, EuroBank can easily classify DLSw+ traffic based on the well-known TCP port and can mark it with the EuroBank DSCP of the corresponding traffic class.

The traffic from Automatic Teller Machines (ATMs) that needs to be classified in the Mission-Critical Interactive service class is classified based on the fact that it is encapsulated as X.25 over TCP (XOT) (see [XOT]). Example 6-5 shows the configuration of a local QoS policy that is used to apply a QoS policy to traffic generated by the router itself, as is the case here for the XOT traffic carrying flows from the ATMs.

Example 6-5. Classification and Marking of ATMs Using a Local Policy

!
ip local policy route-map XOT
!
route-map XOT permit 20
match ip address 141
set dscp 26
!
access-list 141 permit tcp any any eq 1998
access-list 141 permit tcp any eq 1998 any
!

PhoneNet uses call managers and IP phones to support the Managed Voice Service offered to EuroBank. The call manager controls which DSCP value the IP phones use in packets carrying VoIP media streams. It also controls which DSCP value the IP phones use as well as the call manager itself in voice signaling packets. This is achieved by first configuring the DSCP values to be used on the call manager. Then the call manager instructs the IP phones, via signaling, to use those values. EuroBank indicated to PhoneNet that DSCP 46 and DSCP 26 are to be used for media stream and voice signaling traffic, respectively. The IP phones and call manager automatically mark those as belonging to the EuroBank Voice and Mission-Critical Interactive service classes without EuroBank's having to classify this traffic.

EuroBank routers automatically mark routing traffic with precedence 6, which corresponds to the DSCP=48 used by EuroBank for the Network Control service class. In the MPLS core, routing traffic is automatically marked with EXP=6 because the 3-bit precedence field gets copied by default into the EXP field at MPLS imposition.

If, based on observations, EuroBank realizes that a given application is not getting appropriate treatment, it can move that application to a different service class. This simply requires that the classification function be adjusted to mark the DSCP of the corresponding flows to the DSCP value of the new service class. As soon as this is done, the corresponding traffic is automatically granted the DiffServ treatment for that service class.

Sub-100-Mbps QoS Policy

EuroBank applies a fine-grain QoS policy involving six queues to manage the congestion that occurs fairly routinely on sub-100-Mbps links. Table 6-5 lists these six queues, the service classes that are mapped into each queue, and the queue parameter configuration.

The corresponding QoS policy is illustrated in Figure 6-24.

It is worth remembering that the router packet scheduler enforcing this QoS policy has a property called work-conserving. This means that bandwidth never gets wasted. As a consequence, if a class does not use all its allocated minimum bandwidth at one time, the unused fraction can be used by other classes that have more to transmit than their own allocated minimum bandwidth. This is why a generous allocation of 30 percent to the Mission-Critical Interactive service class (while the expected maximum load is about 10 percent) is not a wasteful approach. If there happens to be more mission-critical interactive traffic than expected (say 20 percent instead of 10 percent), the corresponding queue has enough capacity to maintain the required high levels of service. If there is no more than the expected load of mission-critical interactive traffic, the bandwidth allocated but unused by the corresponding queue can be reused by the other queues anyway.

This policy is applied in the EuroBank network on all the links whose rate is strictly less than 100 Mbps:

POP-to-POP links based on ATM PVC in the 1040-Mbps range, on E3 links as well as on subrate (40-Mbps) Fast Ethernet links

Office-POP links based on leased line in the 834 Mbps range

The configuration for the six-queue sub-100-Mbps QoS policy is shown in Example 6-6 in the case of a leased-line link.

Example 6-6. Six-Queue Sub-100-Mbps QoS Policy for Leased Line

!
class-map match-any class-EF
match mpls exp 5
class-map match-any class-AF4
match mpls exp 4
class-map match-any class-AF3
match mpls exp 3
match mpls exp 6
match dscp 48
class-map match-any class-AF2
match mpls exp 2
class-map match-any class-AF1
match mpls exp 1
match dscp 8
!
policy-map 6Q-policy
class class-EF
priority percent 20
class class-AF4
bandwidth percent 10
class class-AF3
bandwidth percent 30
class class-AF2
bandwidth percent 20
class class-AF1
bandwidth percent 15
class class-default
bandwidth percent 5
!
int serial1/0/0
service-policy output 6Q-policy

Description of the Metro Connections in the UK," input policing is performed by the operator of the subrate Fast Ethernet service.) This is achieved using hierarchical service policies, which allow a service policy to invoke another service policy as the action to perform on one of its classes. As illustrated in Example 6-7, EuroBank first applies a top-level policy enforcing the aggregate shaping to all the traffic collectively. Then it calls a lower-level service policy that is the same as in Example 6-6 and enforces the six-queue scheduling over the shaped rate.

Example 6-7. Six-Queue Sub-100-Mbps QoS Policy for Subrate Fast Ethernet

!
policy-map 6Q-policy-with-40Mbps-shaping
class class-default
shape average 40000000
service-policy 6Q-policy
!
interface fastethernet0/0
service-policy output 6Q-policy-with-40Mbps-shaping

In the case of ATM PVCs, the QoS policy first needs to apply ATM-level traffic shaping in accordance with the contracted ATM traffic class (VBR-nrt) and the contracted sustainable cell rate and peak rate. Then it needs to apply the scheduling policy independently within each ATM PVC. Examples of application of QoS policies over ATM PVCs can be found in the section "QoS Design in the Core Network on ATM PVCs" in Chapter 5, "Global Service Provider Design Study."

100+ Mbps QoS Policy

A simpler three-queue policy is applied on 100+ Mbps interfaces. On these interfaces, congestion is rare, not very heavy, and relatively short. Thus, it isn't necessary to fine-tune the congestion management mechanisms for each service class. The key objectives during the short congestion periods are simply to ensure that the voice traffic remain completely protected and that the applications that can afford to be slowed down are indeed squeezed to make room for all the interactive and important applications. This is achieved through the policy whose queues, service class mapping, and queue configuration are presented in Table 6-6.

This policy is applied in the EuroBank network on all the links whose rate is in the 100155 Mbps range:

POP-to-POP full-rate Fast Ethernet and STM-1 links

Office-POP STM-1 links

Data center-POP full-rate Fast Ethernet and STM-1 links

The three- queue QoS policy is illustrated in Figure 6-25.

Gigabit Ethernet Link QoS Policy

Because subrate (300600-Mbps) Gigabit Ethernet links are completely uncongested, EuroBank decided not to implement any QoS mechanisms on these links. It might revisit that decision in the future should congestion and noticeable QoS degradation appear on those links.

QoS Design on the Access for Branches

EuroBank branches are connected to the EuroBank core through a managed Layer 3 MPLS VPN service or a managed Frame Relay-based router service. Therefore, EuroBank service classes need to be mapped onto the classes of service supported over each of these services. This section details the case of interoperation with the managed Layer 3 MPLS VPN service used to access branches in Germany, but a very similar approach is followed in Spain and the UK.

The Layer 3 MPLS VPN operator supports three classes of service (CoSs). These classes of service are listed in Quality of Service Design" section in Chapter 4.

Traffic Flowing from a Branch

This section considers traffic flowing from a branch toward the EuroBank core.

Because the LAN within the branch is exclusively based on Fast and Gigabit Ethernet switching that is overprovisioned compared to the current load, no QoS mechanisms need to be activated within the branch itself.

Traffic leaving the branch is first classified and marked by the service provider managed CE router. EuroBank used the web-based interface provided by the service provider to configure the classification criteria that need to be applied by the CE router. This is done on EuroBank's behalf to classify traffic into the three service provider classes of service in accordance with the mapping of Table 6-7. For instance, EuroBank indicated that the CE router needs to identify packets carrying the CITRIX protocol and classify those in the Premium CoS. As another example, because VoIP media streams as well as VoIP signaling messages are premarked by the IP phones, EuroBank indicated to the service provider that the packets arriving with DSCP=46 (EuroBank's DSCP value for voice) need to be mapped to the Real-Time CoS. Packets arriving with DSCP=26 (EuroBank's DSCP value for Mission-Critical Interactive, which includes VoIP signaling) need to be mapped to the Premium CoS.

When marking traffic after classification, the service provider uses its own DSCP values (as opposed to EuroBank's DSCP values).

This classification and marking, as well as the subsequent QoS steps involved in proper end-to-end QoS handling in the case of traffic flowing from a branch, are illustrated in Figure 6-26.

The service provider relies on the DSCP field being marked to its own values to apply its corresponding QoS mechanisms. These QoS mechanisms are applied on the CE router located in the branch for transmission onto the access link, as well as at every hop in the service provider's network, including eventually on the PE router for transmission onto the link toward EuroBank. This ensures that traffic experiences QoS performance levels in accordance with the SLA commitments provided by the service provider for each of its three classes of service from the CE router in the branch to its delivery point into the EuroBank network (on input into the EuroBank PE router).

Because the EuroBank PE router receives packets from the service provider network marked as per the service provider DSCP scheme, EuroBank applies an input policy on that interface that maps the DSCP values back to EuroBank's DSCP values. This mapping is based on Table 6-7 and is illustrated in Example 6-8. The packets received and marked as Premium are mapped into EuroBank's Mission-Critical Interactive service class regardless of whether they were marked by the service provider as in-contract or out-of-contract. The fact that some packets crossed the policed contract rate enforced by the service provider does not mean that such packets are less important to EuroBank.

Example 6-8. DSCP Mapping Policy from the Service Provider to EuroBank

!
class-map match-any class-RealTime-SP
match dscp 40
class-map match-any class-Premium-SP
match dscp 24
match dscp 16
!
policy-map DSCP-Mapping-SP-to-EuroBank
class class-RealTime-SP
set dscp 46
class class-Premium-SP
set dscp 26
class class-default
set dscp 0
!
int pos0/0
service-policy input DSCP-Mapping-SP-to-EuroBank

Because the service provider supports only three classes of service, the traffic from the branch is mapped into only three of the nine EuroBank service classes.

EuroBank considered performing fine-grain classification on all the traffic received from the service provider (instead of just based on the received DSCP). It needed to be able to classify the traffic into its full range of service classes (instead of into only three of those). However, EuroBank did not retain that approach for a number of reasons. First, it may have a performance impact because it applies to the aggregate traffic received from all the branches in that country. Second, the branches run only a subset of the EuroBank service classes because, for example, multicast and videoconferencing are not currently used in branches. Finally, a coarse differentiation of voice, mission-critical, and default addresses the essential QoS requirements.

As soon as packets have their DSCP marked according to EuroBank DSCP values, they are handled accordingly by the QoS policies applied inside the EuroBank network.

Traffic Flowing to a Branch

This section considers traffic flowing from the EuroBank core toward a branch through the service provider Managed Router service.

This time packets get classified and marked by EuroBank. Therefore, they are initially marked with DSCP values from EuroBank scheme and experience the corresponding treatment throughout the EuroBank network.

For the packets to be handled appropriately through the service provider network, EuroBank applies an egress policy on its egress PE router. It remarks the packet DSCP according to the service provider DSCP values, as illustrated in Example 6-9. This egress policy is also the one applying the three-queue QoS policy on the link toward the EuroBank Layer 3 MPLS VPN network.

Example 6-9. DSCP Mapping Policy from EuroBank to the Service Provider

!
class-map match-any class-EF
match dscp 46
class-map match-any class-AF4
match dscp 26
match dscp 48
match dscp 18
match dscp 10
!
policy-map 3Q-policy-with-DSCP-Mapping-EuroBank-to-SP
class class-EF
set dscp 40
priority percent 20
class class-AF4
set dscp 24
bandwidth percent 75
class class-default
set dscp 0
bandwidth percent 5
!
int pos0/0
service-policy output 3Q-policy-with-DSCP-Mapping-EuroBank-to-SP

The application point for this DSCP mapping policy, as well as the subsequent QoS steps involved in proper end-to-end QoS handling in the case of traffic flowing to a branch, are illustrated in Figure 6-27.

After being received with appropriate DSCP marking by the service provider PE router, packets are handled according to the three classes of service scheme by the service provider. This includes corresponding scheduling by the service provider PE router onto the access link toward the branch.

Finally, packets are forwarded by the CE router onto the branch LAN. Because no QoS mechanisms are used inside the LAN, it does not matter that packets are transmitted with DSCP values according to the service provider scheme instead of the EuroBank scheme. If EuroBank were to apply QoS policies within the branch, one approach could be to get the service provider CE router to map back the DSCP values into EuroBank values. (This is very much like the DSCP mapping performed by the EuroBank PE router for traffic flowing from the service provider into the EuroBank core, as shown in Example 6-8.)