|
MEF E-Tree Service Over MPLS
Needs, Myths And Challenges
Content Disclaimer
ABSTRACT
Ethernet services have been increasingly deployed by carriers over the last few years. The most successful so far are E-Line (point-to-point) and E-LAN (multipoint-to-multipoint) services. To achieve carrier grade objectives, such deployments have been done over MPLS backbones. Recently though, carriers have seen the need for deploying a new type of Ethernet services called “E-Tree” over their infrastructure. This type of services is expected to complement the range of Ethernet products offered by carriers today with new possibilities and new applications, specifically in the distribution of content (typically video), mobile backhaul and clock synchronisation to only mention a few. However, with any new type of services, new challenges also appear in relation to operational deployment.
This article highlights the different use cases and potential benefits of deploying E-Tree services over an MPLS backbone. It also presents some of the short term and long term challenges facing such deployments and some of the elements currently under development in the Internet Engineering Task Force (IETF) to address these specific challenges.
INTRODUCTION
Ethernet has evolved from a technology developed for Local Area Network (LAN) environments to a technology used in Carrier’s networks.
Carriers have over the last few years started offering a variety of Ethernet services with the two most popular being the E-Line (point-to-point services) and E-LAN (multipoint-to-multipoint services) defined by the Metro Ethernet Forum (MEF) [1]. Carriers in their vast majority have taken the approach to deploy or reuse their multiprotocol label switching (MPLS) networks to offer such services. The building block for encapsulating Ethernet frames over an MPLS network is a specific entity called a pseudowire (PW). The Internet Engineering Task Force (IETF) has published several request for comments (RFCs) detailing how such Ethernet services can be delivered over MPLS networks [2] [3] [4].
Figure 1. MEF Ethernet Service Types: E-Line, E-LAN and E-Tree.
Recently, carriers have started to see the need to offer a new type of multipoint Ethernet services defined by MEF, called E-Tree [1]. At the difference of E-Line and E-LAN where there are no communication restrictions between endpoints, on E-Tree each endpoint is designated as either a root or a leaf. Root can communicate with all other endpoints on the E-Tree, however leaf can only communicate with roots but not leafs. There may be single root or multiple roots in one E-Tree service. Carriers are looking at leveraging their current MPLS infrastructure to deploy E-Tree services.
This article discusses the relevant use cases for E-Tree services as well as the immediate challenges and potential deployment options.
USE CASES
Because of their specific topology, E-Tree service constructs are very well suited for content delivery applications. Typically, one or several root endpoints will be responsible for delivering some sort of content to multiple leaf endpoints. Some of the scenarios are detailed further in this section but, broadcast video and clock synchronisation are some of the applications that could make use of such a service construct. Because in those scenarios, the root endpoints will distribute the same information to many leaf endpoints, there is a strong potential for bandwidth optimisation throughout the carrier’s network. Bandwidth optimisation mechanisms allow for a carrier to reduce the amount of traffic that needs to go through its physical infrastructure and therefore reduce deployment costs.
Interestingly, other applications may not benefit greatly or at all from bandwidth optimisation but will require the leaf-to-leaf communication restriction of E-Tree constructs. For example, applications such as Internet access or hub & spoke virtual private networks (VPNs) may have no need for bandwidth optimisation but definitely have stringent security requirements in terms of which endpoints are not allowed to communicate directly. This security aspect for E-Tree also leads to another very important business application for E-Tree services which is their specific use for wholesale offerings.
Typically, instead of deploying their own infrastructure, some carriers (or some application/content service providers with no network infrastructure at all) that need to extend the reach of their services may rely on another carrier’s infrastructure covering a specific area. This means that carriers only need to interconnect with other carriers at some specific key locations called points of interconnect (POI) and rely on someone else infrastructure to deliver their services to their customers. The carrier that owns the infrastructure (or called the infrastructure carrier) is therefore “wholesaling” network services to another carrier. This is particularly relevant for the access and backhaul part of the network.
Figure 2. E-Tree for Wholesale Access
Please also note that some applications may benefit from both the bandwidth optimisation and security aspects of E-Tree service constructs. This would for example be the case for a hub & spoke VPN for franchise business which requires separation between franchisees and has broadcast video as one of the major applications on the VPN.
Figure 3. E-Tree for Hub & Spoke VPN
The rest of this section details some of the proposed applications that could be deployed over E-Tree constructs.
BROADCAST VIDEO
In this use case, a video head-end provides a service to multiple subscribers. The video head-end is the root of the E-Tree service while each subscriber represents a leaf. The video head-end broadcasts a set of channels to all subscribers.
Figure 4. E-Tree for Broadcast Video
This type of E-Tree service construct is unidirectional with only root to leaf communication and no leaf to root traffic. Because every time the root sends a frame, all leafs on the E-Tree will receive it, there is a potential to offer some bandwidth optimisation mechanisms.
Furthermore, if service redundancy is required, it is possible for the carrier to include one or more extra root endpoints to the E-Tree construct and ensuring that only one of these root endpoints transmits data into the E-Tree.
If subscribers need to send control messages back to the video head-end, or fast channel change (FCC) is required, then the E-Tree construct becomes bidirectional.
The first type of E-Tree construct (unidirectional traffic only) is particularly relevant to initial deployments of E-Tree services. IETF Layer 2 Virtual Private Networks Working Group is currently developing solution Virtual Private Multicast Service (VPMS) for both models (unidirectional and bidirectional) [5].
CLOCK SYNCHRONISATION
This use case is about clock synchronisation using IEEE 1588 Precision Time Protocol (PTPv2). In the E-Tree construct, the PTP server is the root while each PTP client is a leaf. There may be multicast traffic from PTP server to PTP clients and unicast traffic between PTP server and PTP client, but no traffic between PTP clients. For redundancy, it is possible to have more than one root endpoints to support multiple PTP servers.
Figure 5. E-Tree for Clock Synchronisation
This type of E-Tree service construct is bidirectional with both root to leaf and leaf to root communication. The traffic from root could be towards a single leaf or towards all leafs.
MOBILE BACKHAUL
In mobile backhaul, there are two main elements, the radio access network base stations (RAN BS) and the RAN network controller (RAN NC). The RAN BS sites only need to exchange data with RAN NC sites. Latest models of RAN NCs and RAN BSs support native Ethernet interfaces, therefore an E-Tree service construct where the RAN NC is a root and the RAN BSs are leafs could be used here.
Figure 6. E-Tree for Mobile Backhaul
This type of E-Tree service construct is bidirectional with both root to leaf and leaf to root communication. Furthermore, there is also a mix of Ethernet traffic that will transit through this service construct. The traffic from root could be towards a single leaf or towards all leafs (clock synchronisation using IEEE1588 PTPv2 for example).
INTERNET ACCESS
The most popular deployment of Internet access relies on E-Line based topology where a gateway router has one point-to-point logical interface towards each router deployed at the customer premises. The reason behind such design is the required security separation between customers. This means that the gateway router has to maintain a large number of logical interfaces (one for each remote endpoint).
One alternative here would be to use an E-Tree service construct with the gateway router as a root and all the routers located on the customer premises as leafs. The two key points of E-Tree here are to provide provisioning/maintenance simplification compared to the existing E-Line case while still maintaining the security separation between customers. Another advantage is the possibility to offer redundancy by adding a second (or more) gateway router as another root endpoint without having to duplicate all of the provisioning as in the E-Line case.
Figure 7. E-Tree for Internet Access
This type of E-Tree service construct is bidirectional with both root to leaf and leaf to root communication. Here there is no root to all leafs traffic, all traffic is always point to point between the root and a specific leaf.
The four previous use cases are some examples of how an E-Tree service could be used. As mentioned at the beginning of this section there are also other applications that would benefit from the E-Tree services (such as hub & spoke VPNs, wholesaling, etc).
The important point to notice here, is that even though the Ethernet service constructs are the same (e.g. there is one or more roots, one or more leafs and no leaf-to-leaf communication) the key requirements that a particular E-Tree service must fulfil will change based on its application. In some cases efficient broadcast delivery is critical but security is not really a concern (e.g. broadcast video), in some other cases efficient broadcast delivery is not really needed as opposed to making sure that leafs do not talk to each other (e.g. Internet access).
The question for carriers is therefore whether there is a single “fit-all” type of E-Tree or a few variants are required. In order to address this question we must look at some of the myths around E-Tree services.
SOME MYTHS
There are several myths generally associated with E-Tree constructs. Four of them are discussed here. The first three are around the generic E-Tree service definition; the fourth one is about the core topic of this article, their deployment over MPLS networks.
FIRST MYTH: DELIVERY METHOD IS THE SAME AS TRADITIONAL ETHERNET
The first myth usually associated to E-Tree is that a service frame in an E-Tree construct must follow the same delivery rule as traditional Ethernet bridged/switched networks with the exception of the leaf-to-leaf communication restriction.
The traditional Ethernet delivery mechanism is MAC based forwarding. In simple words, that is dynamic learning of MAC address at service endpoint based on source MAC address of ingress frame and frame delivery based on destination MAC address in the frame. For known unicast address, deliver to the service endpoint associated with that MAC address, otherwise (i.e. unknown unicast, broadcast or multicast) deliver to all other endpoints.
Let’s look at what is stated in the MEF service definition [1]. For the “Service Frame Delivery” attributes, the requirement is specified as “Deliver Unconditionally or Deliver Conditionally. If Delivered Conditionally, MUST specify the delivery criteria”. This allows a carrier to define the service frame delivery attribute for a particular E-Tree service according to the specific application requirements (e.g. broadcast video, or Internet access, or hub & spoke VPN, etc). Typically, such service frame delivery attribute will be MAC based forwarding same as traditional Ethernet delivery mechanism.
As presented in the previous section, different use cases may impose different requirements. MAC based forwarding may be required in some cases but not always. A typical example is E-Tree construct for broadcast video, which is unidirectional root to all leafs only, therefore MAC based forwarding is not required. This is highlighted in [5].
In some special scenarios, MAC based forwarding may even introduce some security challenges. This will be further discussed in later sections.
SECOND MYTH: E-TREE IS RESTRICTED TO BROADCAST AND MULTICAST TRAFFIC
The second myth is that E-Tree constructs are limited to broadcast and multicast traffic from the root and that they do not need to support unicast traffic at all.
Let’s look at what is stated in the MEF service definition [1]. There are three “Service Frame Delivery” attributes explicitly defined, namely “Unicast Service Frame Delivery”, “Multicast Service Frame Delivery” and “Broadcast Service Frame Delivery”. Clearly, this does not restrict the type of traffic that can transit over an E-Tree construct.
Again, the different use cases presented before show that unicast traffic, broadcast traffic or a mix of unicast, multicast and broadcast traffic can all be valid traffic transiting over an E-Tree construct.
THIRD MYTH: E-TREE IS OPTIMISED FOR BROADCAST AND MULTICAST TRAFFIC
The third myth is that an E-Tree construct is optimised for broadcast and multicast traffic from the root.
Basically, an E-Tree is an Ethernet service construct only. How the traffic is forwarded within the E-Tree will depend on the network construct.
If we look at the example of broadcast video in Figure 8, when using E-Line (left), the video source will replicate the video content and send one copy per end user to the network. This approach does not help with bandwidth optimisation.
Figure 8. Bandwidth Optimisation for Broadcast Video
In contrast, using E-Tree (center), the video source will only send one copy to the root endpoint and let the network do the replication and distribution to all leaf endpoints. There is therefore an opportunity for bandwidth optimisation inside the network. However, even with the E-Tree construct, bandwidth optimisation may not always be achieved.
There are at least two further options to construct an E-Tree. In option 1 (top right) the E-Tree construct is based on point-to-point (P2P) network constructs and packets are replicated at the ingress endpoint towards every leaf. Like in the E-Line case, this does not really bring much bandwidth optimisation (at the exception of the traffic between the video source and the root endpoint).
With option 2 (bottom right) the network construct is point-to-multipoint (P2MP) and packets are only replicated at optimal fan-out points on the path to all leafs. This approach brings full potential for bandwidth optimisation.
FOURTH MYTH: E-TREE OVER MPLS IS EASY AND READY TO GO – JUST RE-USE VPLS WITH MINOR CHANGES
The final myth is that E-Tree deployment over MPLS infrastructure is easy and ready to go by just reusing the E-LAN design using VPLS with minor changes.
Figure 9. E-Tree over MPLS - reuse E-LAN design using VPLS
Figure 9 presents a specific scenario of E-Tree where the combination of the current VPLS and PW implementation is not sufficient for the leaf-to-leaf communication restriction.
On the left, an E-LAN design using Virtual Private LAN Service (VPLS) is given. Each edge router holds a Virtual Switch Instance (VSI) that emulates an Ethernet switch. All these VSIs are fully meshed together via point-to-point PWs so that they can communicate with each other. One further constraint added (to avoid loops) and detailed in [3] & [4] is that an edge router must not forward traffic from one PW to another PW in the same VPLS mesh.
In the center of Figure 9, a simple E-Tree construct with a single root is presented. To fulfil the leaf-to-leaf communication restriction, some PWs between VSIs have been removed. Furthermore, an extra constraint (split horizon group) is added on each edge router to ensure local leafs and PWs to remote leafs do not talk to each other. With these minor changes, we can successfully build an E-Tree service by reusing the VPLS standards [3] & [4].
The first challenge will arise when several nodes have both root and leaf. This is presented on the right in Figure 9. Edge router PE1 does not know whether a frame received from edge router PE2 via the PW is from a root or a leaf on PE2 and therefore whether the frame can be delivered to a local leaf or not.
Even though there are options around this issue (like only having a single root for the E-Tree or attaching leafs and roots to different network elements), none of these options present a generic solution to the overall problem.
The second challenge that can be seen from Figure 9 is that with standard VPLS implementation, each edge router is responsible for replicating the ingress traffic via the use of point-to-point PWs. As explained in the previous section, such a construct is not optimised for a mix of point-to-point and point-to-multipoint traffic.
The third challenge relates to security issues in multi-subscriber scenarios, such as MAC address spoofing and unknown unicast delivery.
CHALLENGES
EGRESS ROUTER DOES NOT KNOW A FRAME’S ORIGIN ON AN INGRESS ROUTER
One of the key benefits but also a key challenge of E-Tree services is the leaf-to-leaf communication restriction. As explained in the previous section, when two or more edge routers have both root and leaf endpoints, in some situations an egress router does not know whether a frame received on a PW is from a root or a leaf on the ingress router and therefore has difficulty in enforcing the leaf-to-leaf communication restriction. Several options to address this challenge are discussed here.
Derived from Source MAC Address in the Frame
In the current VPLS implementation, the egress router does not know which endpoint on the ingress router a source MAC address belongs to. Additional information exchange between edge routers is required, either peer to peer or through operation support system (OSS). This involves signalling protocol development or system development, both not simple tasks. On the other hand, due to the dynamic nature of MAC address to endpoint mapping, this approach is not considered ideal for security.
Static MAC Address for Root Endpoint
For each root endpoint, static MAC addresses are configured and MAC learning is disabled. A full list of root static MAC addresses is maintained in every edge router. As anti-spoofing control, at root endpoint only permit ingress frame with source address equal to one of the static MAC addresses of that root, at leaf endpoint deny any ingress frame with source address equal to any of the root static MAC addresses.
The decision on whether a frame is from root or leaf becomes very simple. If a source MAC address matches a static root MAC address, this means that the frame is from a root endpoint, otherwise the frame is from a leaf endpoint.
Obviously, the drawback of this approach is that more configuration changes will be required. However, if we look at the use cases, the devices connected to root endpoints are not changed often (e.g. Internet gateway router, RAN NC, PTP Server, video server) and therefore static MAC addresses for root endpoint is considered feasible in most cases.
This is considered a simple and ready to go option for immediate E-Tree deployment.
Hierarchical VPLS Approach
The Hierarchical-VPLS (H-VPLS) approach relies on a single node doing the split horizon rule (enforce the leaf-to-leaf communication restriction). Basically, all leafs and roots are transported back to a single node that will then apply locally a split horizon rule.
The first drawback of this approach is that the node doing the split horizon becomes a single point of failure for the E-Tree service. This can be mitigated by using two nodes with split horizon rule and using PW redundancy but obviously with the price of complexity. The second drawback is traffic hair pinning, which potentially cause increase in bandwidth utilisation and also end-to-end latency.
Carry a “From Root or Leaf” Flag in the Frame
One potential long term solution is to carry a “from root or leaf” flag per frame on the PW. Currently there is no protocol standard on where within the frame this flag should be, how the ingress router should set it and how the egress router should act upon it.
One option under consideration is the use of reserved bits in the PW Control Word. This approach requires simple extensions to current PW and VPLS protocol standards. Relevant Internet drafts are [6] [7] [8] [9].
Other Solution Approach
A different solution based on “Asymmetric VLAN Shared Learning” has also been proposed. This approach also requires extensions to current standards. Relevant Internet draft is [10].
Refer to IETF L2VPN working group mailing list for collaboration happening in IETF.
BANDWIDTH OPTIMISATION
As can be seen from the use cases, some of the E-Tree constructs can be used to distribute the same “content” from a root to many leafs. This brings a specific challenge to the carrier which is about bandwidth optimisation.
Under the current VPLS and PW standards developed by the IETF, each edge router of an MPLS domain that needs to connect to one or more other edge routers for an E-Tree construct will use a point-to-point (P2P) PW to connect to each remote router. The edge router (where the root is connected) is responsible for replicating towards all the edge routers (where leafs are connected) every Ethernet broadcast/multicast frame coming from the root.
Network Optimisation Options
Figure 10 highlights the network scenario where bandwidth optimisation for broadcast is possible.
Figure 10. Bandwidth Optimisation for Broadcast
On the left, the PWs are transported on totally different physical paths. In this case no further bandwidth optimisation can be achieved.
However, in the center, a different physical network topology where the four edge routers are connected via a fifth router (P) in the middle, we can see that the three PWs transit via the same physical link between PE1 and P, which means multiple copies of the same content are transported on the physical link. In this case, there is a potential for bandwidth optimisation in the carrier network.
To address this gap in the current PW toolkit, the IETF has started working on the concept of point-to-multipoint (P2MP) PWs [11] [12]. For simple deployments, the P2MP PWs can be unidirectional, but the IETF is also developing bidirectional P2MP PWs.
As shown in the right, with P2MP PWs, the edge router is no longer responsible for replicating towards all of the egress edge routers. The network elements along the physical path (router P in this example) can now participate in the replication process. This is achieved via the use of a P2MP PW with the replication done by the underlying layer via a P2MP label switched path (LSP).
Even though the prospects of P2MP PWs are attractive to carriers, it is worth mentioning that at the time of this article, some long term enhancements are still under development (such as dynamic setup, root and leaf redundancy, monitoring). This means that in the short term carriers will have to rely on either the classical P2P approach or a simplified P2MP PW approach that is fully static and manually configured. Relevant RFC and Internet drafts are [13] [14] [15].
SECURITY IN MULTI-SUBSCRIBER ENVIRONMENT
Some of the important security aspects for an E-Tree service construct have been highlighted in the Internet access case. This section discusses some of the challenges related to MAC based forwarding in a multi-subscriber environment and how they can be addressed in the short term and long run.
The first issue is that sensitive information belonging to a specific customer may also be delivered to another customer’s location, as illustrated in Figure 11. This is due to the unknown unicast delivery mechanism (i.e. broadcast to all when don’t know where to forward) in standard MAC based forwarding.
Figure 11. Unicast Frame from Root to Leaf.
The second issue is the well known problem of MAC address spoofing, which also may result in sensitive information belonging to one customer being delivered to another customer’s location. This is due to the MAC address learning mechanism in standard MAC based forwarding.
These issues are only relevant when different customers, mutually un-trusted, are connected to leaf endpoints in the same E-Tree service (e.g. Internet access). This can be mitigated by inserting a carrier controlled router between the customer network and the leaf endpoint but it is not always feasible.
Immediate requirements and Long term evolutions required
One possible option for immediate deployment is the use of static MAC addresses for leaf endpoints instead of the dynamic MAC address learning. This will resolve both issues mentioned above. Obviously, the major drawback is the manual configuration that needs to happen. However, as we have explained with the use cases, not all E-Tree are multi-subscriber environment, and in most cases there are only very limited number of MAC addresses per leaf endpoint (typically one when a customer router is connected to the leaf endpoint).
The long term approach would be to develop a frame delivery method “more secure” than the traditional Ethernet delivery of MAC based forwarding. One option to consider would be the use of VLAN ID as delivery criteria.
CONCLUSIONS
This article has focused on the potential benefits and challenges of deploying E-Tree services over an MPLS infrastructure. E-Tree services appear as good candidates for a wide range of applications, from content distribution applications and internet access to wholesale products. The promise is that E-Tree services will ease provisioning, monitoring, support as well as wholesaling of Ethernet products and will complement the range of Ethernet products offered by carriers today (mainly E-Line and E-LAN) with new offerings.
However, this wide range of use cases brings a set of different requirements on the E-Tree construct, with potentially several types of E-Tree services having to address very different requirements on the same MPLS network. For example, video distribution and Internet access main requirements (one is about bandwidth optimisation, the other about security) will drastically differ but both will benefit from an E-Tree service delivery.
On top of that, this article has highlighted some of the short term and long term challenges in the way of successful E-Tree deployments. While the IETF is currently addressing some of the long term elements with the development of VPMS and P2MP PWs, carriers will most likely have to rely on some options using static and manual processes in the short term.
Some initial thoughts for long term development are also presented in this article, but what this means is that further work is needed. Collaboration between carriers and vendors as well as extra effort in the different standard bodies are required before carriers can claim that they have truly reached a single converged Carrier Ethernet architecture.
Disclaimer: The views expressed in this article are the authors’ own views only and do not reflect the views of the authors’ employers.
ACKNOWLEDGEMENTS
The authors would like to thank Lizhong Jin, Lucy Yong, Wim Henderickx and Yuji Kamite for their valuable input and support.
REFERENCES
[1] Metro Ethernet Forum, “Ethernet Services Definitions – Phase 2”, MEF6.1, April 2008.
[2] Martini, et al., “Encapsulation Methods for Transport of Ethernet over MPLS Networks”, RFC4448, April 2006.
[3] Kompella & Rekhter, “Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling”, RFC4761, January 2007.
[4] Lasserre & Kompella, “Virtual Private LAN Service (VPLS) Using Label Distribution Protocol (LDP) Signaling”, RFC4762, January 2007.
[5] Kamite, et al., “Framework & Requirements for Virtual Private Multicast Service (VPMS)”, draft-ietf-l2vpn-vpms-frmwk-requirements-02, work in progress, October 2009.
[6] Key, et al., “A Framework for E-Tree Service over MPLS Network”, draft-key-l2vpn-etree-frwk-02, work in progress, March 2010.
[7] Key, et al., “Requirements for MEF E-Tree Support in VPLS”, draft-key-l2vpn-vpls-etree-reqt-00, work in progress, April 2010.
[8] Key, et al., “Extension to VPLS for E-Tree”, draft-key-l2vpn-vpls-etree-02, work in progress, January 2010.
[9] Delord, et al., “Control Word Reserved bit for use in E-Tree”, draft-delord-pwe3-cw-bit-etree-02, work in progress, January 2010.
[10] Jiang, “VPLS PE Model with E-Tree Support”, draft-jiang-l2vpn-vpls-pe-etree-00, work in progress, March 2010.
[11] Jounay, et al., “Requirements for Point-to-Multipoint Pseudowire”, draft-ietf-pwe3-p2mp-pw-requirements-02, work in progress, January 2010.
[12] Martini, et al., “Signaling Root-Initiated Point-to-Multipoint Pseudowires using LDP”, draft-martini-pwe3-p2mp-pw-01, work in progress, October 2009.
[13] Kamite, et al., “Requirements for Multicast Support in Virtual Private LAN Services”, RFC5501, March 2009.
[14] Raggarwa, Kamite & Fang, “Multicast in VPLS”, draft-ietf-l2vpn-vpls-mcast-06, work in progress, March 2010.
[15] Delord, et al., “Extension to LDP-VPLS for Ethernet broadcast and multicast”, draft-delord-l2vpn-ldp-vpls-broadcast-exten-01, work in progress, May 2010.
|
