| Internet-Draft | Seamless SR Problem Statement | September 2021 |
| Hegde, et al. | Expires 28 March 2022 | [Page] |
This draft documents a set of use cases and requirements for end-to-end intent-based paths spanning multi-domain packet networks. The document explicitly focuses on use cases that require high scale and availability, which will likely benefit from distributed solutions. It is intended that the requirements in this document serve as a basis for future IETF work to develop distributed solutions for inter-domain intent-based transport paths.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 28 March 2022.¶
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶
Evolving trends in wireless access technology, cloud applications, virtualization, and network consolidation all contribute to the increasing demands being placed on a common packet network. In order to meet these demands, a given network will need to scale horizontally in terms of its bandwidth, absolute number of nodes, and geographical extent. The same network will need to scale vertically in terms of the different services that it needs to simultaneously support.¶
In order to operate networks with large numbers of devices, network operators organize networks into multiple smaller network domains. Each network domain typically runs an IGP which has complete visibility within its own domain, but limited visibility outside of its domain. Network operators will continue to use multiple domains to scale horizontally. These multi-domain networks will also need to scale vertically, to allow a common multi-domain network to carry all of an organization's services.¶
Evolving network requirements (e.g., 5G, native cloud) motivate network operators to deploy tunnels that span multiple AS's and maintain specific transport characteristics (e.g., bandwidth, latency). There is a need to provide flexible, scalable, and reliable end-to-end connectivity for multiple services across independent network domains.¶
Service Provider networks can contain many nodes distributed over a large geographic area. 5G networks can include as many as one million nodes, with the majority of those being radio access nodes. Radio and access nodes may be constrained by their memory and processing capabilities.¶
Service provider transport networks use multiple domains to support scalability. For this analysis, we consider a representative network design with four level of hierarchy: access domains, pre-aggregation domains, aggregation domains and a core. (See Figure 1). The separation of domains internal to the service provider can be performed by using either IGP or BGP.¶
+-------+ +-------+ +------+ +------+
| | | | | | | |
+--+ P-AGG1+---+ AGG1 +---+ ABR1 +---+ LSR1 +--> to ABR
/ | | /| | | | | |
+----+/ +-------+\/ +-------+ +------+ /+------+
| AN | /\ \/
+----+\ +-------+ \+-------+ +------+/\ +------+
\ | | | | | | \| |
+--+ P-AGG2+---+ AGG2 +---+ ABR2 +---+ LSR2 +--> to ABR
| | | | | | | |
+-------+ +-------+ +------+ +------+
ISIS L1 ISIS L2 ISIS L2
|-Access-|--Aggregation Domain--|---------Core-----------------|
5G networks support a variety of service use cases that require end-to-end slicing. In certain cases the end-to-end connectivity requires the ability to forward over intent-based paths. The inter-domain solution should support end-to-end Service Level Objectives(SLO) to allow the creation of network slices.¶
As WAN networks grow beyond several thousand nodes, it is often useful to divide the network into multiple IGP domains, as illustrated in Figure 2. Separate IGP domains increase service availability by establishing a constrained failure domain. Smaller IGP domains may also improve network performance and health by reducing the device scale profile (including protocol and FIB scale).¶
+-------+ +-------+ +-------+
| | | | | |
| ABR1 ABR2 ABR3 ABR4 |
| | | | | |
PE1+ D1 +-----+ D2 +-----+ D3 +PE2
| | | | | |
| ABR11 ABR22 ABR33 ABR44 |
| | | | | |
+-------+ +-------+ +-------+
|-ISIS1-| |-ISIS2-| |-ISIS3-|
These large WAN networks often cross national boundaries. In order to meet data sovereignty requirements, operators need to maintain strict control over end-to-end traffic-engineered (TE) paths. A goal of a distributed inter-domain solution is to be able to create highly constrained inter-domain TE paths in a scalable manner.¶
Some deployments may use a centralized controller to acquire the topologies of multiple domains and build end-to-end constrained paths. This centralized approach can be scaled with hierarchical controllers. However, there is still significant risk of a loss of network connectivity to one or more controllers, which can result in a failure to satisfy the strict requirements of data sovereignty. The network should have pre-established TE paths end-to-end that don't rely on controllers in order to address these failure scenarios.¶
Data centers are playing an increasingly important role in providing access to information and applications. Geographically diverse data centers usually connect via a high speed, reliable and secure DC WAN core network.¶
+-------+ +-------+ +-------+
| ASBR1 ASBR2 ASBR3 ASBR4 |
| | | DC WAN| | |
PE1+ DC1 +-----+ CORE +-----+ DC2 +PE2
| ASBR11 ASBR22 ASBR33 ASBR44 |
| | | | | |
+-------+ +-------+ +-------+
|-ISIS1-| |-ISIS2-| |-ISIS3-|
In many DC WAN deployments, applications require end-to-end path diversity and end-to-end low latency paths. The DC WAN networks may consist of large number of devices owing to global presence. In some DC WAN deployments the tunneling mechanisms used within the data centers are the same as those used in the DC WAN core. For example, a network may use MPLS in both data center and DC WAN core. Or it may use SRv6 in both data center and DC WAN core. This can simplify network deployments.¶
However, in some DC WAN deployments the traffic within data centers and the traffic over the DC WAN core use different tunneling mechanisms, such as SRv6 in the data center and MPLS in the DC WAN core. It is important for DC WAN network operators to have flexibility in the choice of tunneling mechanisms across domains.¶
The use cases for inter-domain intent-based packet transport described in this section are intended to provide motivation for the requirements that follow.¶
+-----------+ +-----------+ +-----------+
| | | +-+ AS2 | | |
| A1+--+A2 | | A3+--+A4 |
PE1+ AS1 | | |Z| | | AS3 +PE3
| A5+--+A6 | | A7+--+A8 |
| | | +-+ | | |
+--A13--A15-+ +-A17--A19--+ +-----------+
| | | |
| | | |
| | | |
+--A14--A16-+ +-A18--A20--+
| | | |
| A9+--+A10 |
PE4+ AS4 | | AS5 |
| A11+-+A12 |
| | | |
+-----------+ +-----------+
Figure Figure 4 depicts a WAN with multiple ASes. Each AS is resides serves a continent (e.g., Asia). Certain traffic from PE1 (in AS1) to PE3 (in AS3) must not traverse country Z in AS2. However, all paths from AS1 to AS3 traverse AS 2. The inter-domain solution should provide end-to-end path creation that traverses AS 2 but avoids country Z.¶
Service provider networks running L2 and L3VPNs carry traffic for particular VPNs on low-latency paths that traverse multiple domains.¶
+-----------+ +-----------+
| ASBR1 ASBR2 |
| | | |
PE1+ AS1 +----------------+ AS2 +PE2
| ASBR11 ASBR22 |
| | | |
+-----------+ +-----------+
In diagram Figure 5 above, AS1 and AS2 which were previously under independent administration, merge to come under a single administration. The network operator may decide to merge the two domains into single AS which would need bigger operational effort. Network operators may also retain the two ASes and build end-to-end paths across the two Ases. In this case, the paths in AS1 and AS2 corresponding to the same intent may use different representations in the two ASes. In some cases, organizations may continue to use option A or option B [RFC4364] style interconnectivity in which case the inter-domain solution should satisfy intent of the path on inter-domain links for the service prefixes. In other cases, organizations may prefer to use option C style connectivity from PE1 to PE2. In this case, an inter-domain solution should provide effective mechanisms to translate intent across domains without requiring renumbering of the intent representation.¶
[RFC7665] defines service function chaining as an ordered set of service functions and automated steering of traffic through this set of service functions. There could be a variety of service functions such as firewalls, parental control, CGNAT etc. In 5G networks these functions may be completely virtualized or could be a mix of virtualized functions and physical appliances. It is required that the inter-domain solution caters to the service function chaining requirements. The service functions may be virtualized and spread across different data centers attached to different domains.¶
BGP confederation allows the division of a public AS in multiple sub-AS, usually with private identifiers. From outside, the confederation is seen as a single and common AS, the public one. BGP sessions are maintained among sub-AS. In the internals of the confederation, each sub-AS can be configured and run autonomously, even though some BGP parameters (like e.g. LOCAL_PREF or MED) are preserved across sub-AS. Thus, it can be of interest to define end-to-end paths of specific characteristics in terms of SLOs across the sub-AS as well as internally to each sub-AS.¶
Multicast services such as IPTV and multicast VPN also need to be supported across a multi-domain service provider network.¶
+---------+---------+---------+
| | | |
S1 ABR1 ABR2 R1
| Metro1 | Core | Metro2 |
| | | |
S2 ABR11 ABR22 R2
| | | |
+---------+---------+---------+
|-ISIS1-| |-ISIS2-| |-ISIS3-|
Figure 6 shows a simplified multi-domain network supporting multicast. Multicast sources S1 and S2 lie in a different domain from the receivers R1 and R2. Using multiple IGP domains presents a problem for the establishment of multicast replication trees. Typically, a multicast receiver does a reverse path forwarding (RPF) lookup for a multicast source. One solution is to leak the routes for multicast sources across the IGP domains. However, this can compromise the scaling properties of the multi-domain architecture. A distributed inter-domain solution should accommodate a mixture of existing and newer technologies to better facilitate coexistence and migration. A distributed solution should avoid leaking RPF routes into the IGP domains.¶
The requirements described in this document are mostly applicable to network under a single administrative domain that are organized into multiple network domains. The requirements are also applicable to multi-AS networks with closely cooperating administration.¶
This section describes three different ways that multi-domain networks are organized today. The requirements in subsequent sections are applicable to all three types of multi-domain networks described below.¶
----IBGP------EBGP----IBGP------EBGP-----IBGP---
| | | | | |
+-----------+ +-----------+ +-----------+
| | | | | |
| ASBR1+--+ASBR2 ASBR3+--+ASBR4 |
PE1+ AS1 | X | AS2 | X | AS3 +PE2
| ASBR5+--+ASBR6 ASBR7+--+ASBR8 |
| | | | | |
+-----+-----+ +-----------+ +-----------+
PE3
|---ISIS1---| |---ISIS2---| |---ISIS3---|
The above diagram Figure 7 shows three different ASes (AS1, AS2 and AS3.) ASBR1 to ASBR8 are border nodes between the ASes. A given ASBR runs E-BGP sessions with the ASBRs in adjacent ASes. The ASBR also runs I-BGP sessions with other ASBRs in the same AS. Route reflectors can also be used to achieve this full mesh of I-BGP information exchange.Similar scenario applies when considering BGP confederations [RFC5065].¶
----IBGP-----------IBGP-------------IBGP----
| | | |
+-----------+ +------------+ +-----------+
/ \ / \/ \
| ABR1 ABR3 |
| | | |
PE1+ Metro1 + Core + Metro2 +PE2
| | | |
| ABR2 ABR4 |
\ /\ /\ /
+------------+ +-----------+ +------------+
The above diagram Figure 8 shows three different IGP domains, Metro1, Core, and Metro2. The three IGP domains may be realized with multiple levels in ISIS or multiple areas in OSPF. They can also be realized using separate IGP instances.¶
This single-AS network uses I-BGP sessions. ABRs and PEs achieve a full mesh of I-BGP information sharing by configuring the ABRs as inline route reflectors.¶
----IBGP-----IBGP----IBGP------IBGP-----IBGP---
| | | | | |
+-----------+ +-----------+ +-----------+
| | | | | |
| ABR1+--+ABR2 ABR3+--+ABR4 |
PE1+ AS1:D1 | X | AS1:D2 | X | AS1:D3 +PE2
| ABR5+--+ABR6 ABR7+--+ABR8 |
| | | | | |
+-----+-----+ +-----------+ +-----------+
PE3
|---ISIS1---| |---ISIS2---| |---ISIS3---|
The above diagram Figure 9 shows a single AS1 with three different IGP domains, D1, D2, and D3. ABR1 to ABR8 are border nodes for the IGP domains and they participate in only one IGP domain.¶
This single-AS network uses I-BGP sessions. ABRs and PEs achieve a full mesh of I-BGP information sharing by configuring the ABRs as inline route reflectors.¶
The inter-domain solution should support the following unicast tunneling mechanisms:¶
The inter-domain solution should support the ability to have different domains running different unicast tunneling mechanisms.¶
The solution should support inter-domain paths that fulfil a common intent using different unicast tunneling mechanisms in different domains.¶
The inter-domain solution should support the following multicast tunneling mechanisms:¶
The inter-domain solution should support the ability to have different domains running different multicast tunneling mechanisms and should not require to leak RPF routes into IGP domains.¶
The solution should support inter-domain paths that fulfil a common intent using different multicast tunneling mechanisms in different domains.¶
This section discusses the end-to-end constraints that intent-based inter-domain path may have to adhere to. The requirements described in this section are applicable to the three types of AS and IGP domain partitioning described in Section 4.1.¶
Link delay, delay variation and link loss values can be advertised within a domain using the IGP as described in [RFC8570]. Within an IGP domain, minimum latency, minimum delay variation, and minimum link loss paths can be built using flex-algo [I-D.ietf-lsr-flex-algo]. The end-to-end low latency, low delay variation, or low link loss path requires accumulated metrics for latency, delay variation, and link loss.¶
The solution should allow the creation of inter-domain paths with low values of latency as calculated over the end-to-end path. It is not necessary that the solution produce the absolute minimum end-to-end latency, delay variation, or link loss path. However, the solution should provide the ability to balance scalability with optimality.¶
Best path selection at any intermediate border node should be allowed.¶
The inter-domain solution should allow advertising multiple paths end-to-end and compare the accumulated metric across all of the paths at the ingress.¶
A distributed solution should support the creation of inter-domain paths using intra-domain bandwidth guaranteed paths.¶
A distributed solution may support optimized path placement with end-to-end bandwidth guarantees.¶
The links are associated with link-affinity or admin-groups. The link-affinity can be used to indicate a characteristic of a link, such as capacity, encryption, geography, etc. The inter-domain solution should support the creation of paths across different domains that satisfy link inclusion/exclusion constraints. The link constraints should also be satisfied for inter-domain links, such as those between ASBRs.¶
Creating an inter-domain path that includes or excludes a certain set of nodes in each domain should be supported. The inter-domain solution should be independent of the mechanisms used to achieve the node inclusion/exclusion constraints within a domain. For example, an RSVP-based domain may use link affinities to achieve node exclusion constraints, while an SR-based domain may use flex-algo, which natively supports excluding nodes.¶
+-----------+ +-----------+ +-----------+
| | | AS2 | | |
| A1+--+A2 A3+--+A4 |
PE1+ AS1 | | | | AS3 +PE3
| A5+--+A6 A7+--+A8 |
| | | | | |
+--A13--A15-+ +-A17--A19--+ +-AS22--AS23+
| | | |
| | | |
| | | |
+--A14--A16-+ +-A18--A20--+ | |
| | | | | |
| A9+--+A10 AS20---- |
PE4+ AS4 | | AS5 | |
| A11+-+A12 AS21------------
| | | |
+-----------+ +-----------+
Diagram Figure 10 shows a multi-domain, multi-AS network with the possibility for AS-diverse paths. The inter-domain solution should support creation of end-to-end paths that includes/excludes a certain domain entirely. For example, a network operator should be able to use the solution to create a path from PE1 to PE3 that automatically avoids passing through nodes belonging to AS2.¶
The solution should support the creation of node and link-diverse inter-domain paths.¶
The intra-domain portion of the end-to-end paths should make use of existing mechanisms for computing and instantiating diverse paths within a domain.¶
Inter-domain links (such as those connecting ASBRs) should also be taken into account for diverse inter-domain paths.¶
The solution should support SRLG-aware inter-domain diverse paths.¶
Use cases such as data sovereignty described in Section 3.1 require that the paths with certain constraints are applicable to only a subset of domains. In domains where a constraint is not applicable, the end-to-end path should not create any state on the internal nodes.¶
Many of the requirements above are applicable to multicast traffic as well. Some requirements need to be refined with respect to multicast. Multicast also has some unique requirements not shared by unicast. These requirements will be covered in a future version of this document.¶
Seamless MPLS architecture is widely deployed and BGP-LU [RFC3107] is used to connect different domains. The inter-domain solution for intent-based paths should be interoperable with BGP-LU.¶
Options A and B require additional state on border nodes, so they are typically less scalable than option C. However, options A and B can be advantageous when it is necessary to do filtering or policing on border nodes. When option A or B is deployed, the solution should still meet the SLA requirements described in Section 4.3.¶
In cases where two network domains previously under different administrations merge to come under a single administration, it may be preferable to use option C connectivity between the domains. The paths that fulfill the same intent may be represented using different conventions in each domain. The inter-domain solution should support efficient translation of intent from one representation to another.¶
The inter-domain solution for intent-based paths should also provide the ability to create end-to-end best effort paths with accumulated IGP metric across the domains. A deployment should not require two different mechanisms to be deployed for best effort and intent-based paths.¶
As described in Section 4.2.1 and Section 3.6 the inter-domain solution should support one domain having one type of tunneling mechanism and another domain having another type of tunneling mechanism. The different tunneling mechanisms may completely differ in control plane and data plane operations (e.g. SRv6 and MPLS.) The inter-domain solution should provide interoperability between various tunneling mechanisms and provide the ability to create end-to-end intent-based paths.¶
The above requirements focus on the service independent aspects of inter-domain intent-based paths. In order for different services to effectively use these paths, flexible service mapping is required. The sections below summarize the requirements needed to achieve flexible service mapping.¶
The core domain is expected to have more traffic engineering constraints as compared to metros. The ability to map the services to appropriate transport tunnels at service attachment points should be supported.¶
This document focuses on use cases and requirements that may benefit from a distributed solution. Many of these same use cases and requirements can be addressed with centralized approaches or other distributed TE solutions. One example of a centralized approach is described in "Interconnecting Millions of Endpoints with Segment Routing" ([RFC8604]).¶
Distributed and centralized approaches have inherent tradeoffs. Some networks may use a single approach. Other networks may choose to use both distributed and centralized approaches to get the benefits of both. A distributed inter-domain solution should support the requirements below:¶
Many thanks to Kireeti Kompella, Ron Bonica, Krzysztof Szarcowitz, Srihari Sangli, Julian Lucek, Ram Santhanakrishnan, for discussions and inputs. Thanks to Colby Barth, John Scudder, Joel Halpern for review and comments.¶
1.Kaliraj Vairavakkalai¶
Juniper Networks¶
kaliraj@juniper.net¶
2. Jeffrey Zhang¶
Juniper Networks¶
zzhang@juniper.net¶