Internet-Draft BGP CAR Problem Statement November 2021
Rao, et al. Expires 26 May 2022 [Page]
Workgroup:
BESS WorkGroup
Internet-Draft:
draft-dskc-bess-bgp-car-problem-statement-04
Published:
Intended Status:
Informational
Expires:
Authors:
D. Rao
Cisco Systems
S. Agrawal
Cisco Systems
C. Filsfils
Cisco Systems
K. Talaulikar
Cisco Systems
B. Decraene
Orange
D. Steinberg
Lapishills Consulting Limited
L. Jalil
Verizon
J. Guichard
Futurewei
K. Patel
Arrcus, Inc
W. Henderickx
Nokia

BGP Color-Aware Routing Problem Statement

Abstract

This document explores the scope, use-cases and requirements for a BGP based routing solution to establish end-to-end intent-aware paths across a multi-domain service provider network environment.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 26 May 2022.

Table of Contents

1. Introduction

1.1. Objective

This document explores the scope, use-cases and requirements for a BGP based routing solution to establish end-to-end intent-aware paths across a multi-domain service provider network environment.

The targeted design outcome is to define the technology and protocol extensions that may be required in a manner that addresses the widest application.

The problem that the document initially focuses on is the BGP-based delivery of an intent across several transport domains. To do this, it describes existing intent-aware routing solutions that are deployed and then extends the solution scope and architecture to BGP.

The problem space is then widened to include any intent (including NFV chains and their location), any dataplane and the application of the intent-based routing to the Service/VPN routes. All of this is detailed in the rest of the document.

1.2. Color-Aware Routing

Color-Aware Routing (CAR) establishes routed paths that satisfy specific intent in a network. This section describes the basic concepts that define CAR and the protocols that currently support it.

The figure below is used as reference.

              +-----------------------------------+
              |----+                         +----|
              | E1 |                         | E2 |- V/v with C
              |----+                         +----|
              +-----------------------------------+
Figure 1: Color-aware routing reference topology

1.2.1. Intent

Intent in routing may be any combination of the following behaviors:

  • Topology path selection (e.g. minimize metric, avoid resource)
  • NFV service insertion (e.g. service chain steering)
  • Per-hop behavior (e.g. QoS for 5G slice)

An intent-aware routed path may be within a single network domain or across multiple domains.

1.2.2. Color

Color is a 32-bit numerical value that is associated with an intent, as defined in [I-D.ietf-spring-segment-routing-policy]

1.2.3. Colored Service Route

An Egress PE E2 colors a BGP service (e.g., VPN) route V/v to indicate the particular intent that E2 requests for the traffic bound to V/v. The color (C) is encoded as a BGP Color Extended community [I-D.ietf-idr-tunnel-encaps].

1.2.4. Color-Aware Route

(E2, C) is a color-aware route to E2 which satisfies the intent associated with color C.

Multiple technologies already provide color-aware paths in solutions that are widely deployed.

  • SR Policy [I-D.ietf-spring-segment-routing-policy]
  • IGP Flex-Algo [I-D.ietf-lsr-flex-algo]

In the context of large-scale SR-MPLS networks, SR Policy is applicable to both intra-domain and inter-domain deployments; whereas IGP Flex-Algo is better suited to intra-domain scenarios.

1.2.5. Service Route Automated Steering on color-aware route

An ingress PE E1 automatically steers V-destined packets onto a Color-Aware path bound to (E2, C). If several such paths exist, a preference scheme is used to select the best path: E.g. IGP Flex-Algo first, then SR Policy.

1.2.6. Inter-Domain color-aware routing with SR Policy

If E1 and E2 are in different domains, E1 may request an SR-PCE in its domain for a path to (E2, C). The SR-PCE (or a set of them) computes the end-to-end path and installs it at E1 as an SR Policy. The end-to-end color-aware path may seamlessly cross multiple domains.

1.2.7. Need for a BGP-based color-aware routing solution

  • An operator with an existing Seamless-MPLS/BGP-LU inter-domain deployment [I-D.ietf-mpls-seamless-mpls] may prefer a BGP based extension as a more incremental approach
  • There may be an expectation that BGP would support a larger scale
  • Trust boundaries in an inter-domain deployment leads to a preference for a BGP peering based solution

1.2.8. BGP Color-Aware Routing

BGP Color-Aware Routing (CAR) is a new BGP solution which signals intent-aware routes to reach a given destination (e.g., E2). (E2, C) represents a BGP hop-by-hop distributed route that builds an inter-domain color-aware path to E2 for color C.

1.2.9. Architectural consistency among color-aware routing solutions

As seen above, multiple technologies exist that provide color-aware routing in a network. A BGP based solution must be compliant with the existing principles that apply to them:

  • Service routes MUST be colored using BGP Color Extended-Community to request intent

    • V/v via E, colored with C
  • Colored service routes MUST be automatically steered on an appropriate color-aware path

    • V/v via E with C is steered via (E, C)
    • (E, C) provided by any color-aware technology or protocol
  • Color-aware routes MAY resolve recursively via other color-aware routes

    • (E, C) via N recursively resolves via (N, C)

Here is a brief example that illustrates these principles.

     +----------------+  +----------------+  +----------------+
     |                |  |                |  |                | V/v with C1
     |----+          +----+              +----+          +----|/
     | E1 |          | N1 |              | N2 |          | E2 |\
     |----+          +----+              +----+          +----| W/w with C2
     |                |  |                |  |                |
     |    Domain 1    |  |    Domain 2    |  |    Domain 3    |
     +----------------+  +----------------+  +--- ------------+

Figure 2: Color-aware routing inter-domain reference topology

In the figure above, all the nodes are part of an inter-domain network under a single authority and with a consistent color-to-intent mapping:

  • Color C1 is mapped to "low delay"

    • Flex-Algo FA1 is mapped to "low delay" and hence to C1
  • Color C2 is mapped to "low delay and avoid resource R"

    • Flex-Algo FA2 is mapped to "low delay and avoid resource R" and hence to C2

E1 receives two BGP colored service routes from E2:

  • V/v with BGP Color Extended community C1
  • W/w with BGP Color Extended community C2

E1 has the following inter-domain color-aware paths:

  • (E2, C1) provided by BGP CAR which recursively resolves via intra-domain color-aware paths:

    • (N1, C1) provided by IGP FA1 in Domain1
    • (N2, C1) provided by SR Policy bound to color C1 in Domain2
  • (E2, C2) provided by SR Policy

E1 automatically steers the received colored service routes as follows:

  • V/v via (E2, C1) provided by BGP CAR
  • W/w via (E2, C2) provided by SR Policy

The example illustrates the benefits provided by leveraging the architectural principles:

  • Seamless co-existence of multiple color-aware technologies, e.g., BGP CAR and SR Policy

    • V/v is steered on BGP CAR color-aware path
    • W/w is steered on SR Policy color-aware path
  • Seamless and complementary interworking between different color-aware technologies

    • V/v is steered on a BGP CAR color-aware path that is itself resolved within domain 2 onto an SR Policy bound to the color of V/v

1.2.10. Color Domains

  • A color domain represents a collection of one or more network (IGP/BGP) domains with a single, consistent color-to-intent mapping
  • Color re-mapping may happen at color domain boundaries

1.2.11. Per-Destination and Per-Flow Steering with BGP CAR

Ingress PE E1 steers packets destined for a service (VPN) route V/v via BGP Color-Aware Route R/r to E2

  • Per-Destination Steering: Incoming packets on E1 match BGP service route V/v to be steered based on the destination IP address of the packets.
  • Per-Flow Steering: Incoming packets on E1 match BGP service route V/v to be steered based on the combination of the destination IP address and additional elements in the packet header (i.e., IP flow). Such a packet lookup may recurse on a forwarding array where some of the entries are BGP color-aware routes to E2. A given flow is mapped to a specific entry in this array i.e. via a specific BGP color-aware route to E2.

2. Intent bound to a Color

The BGP CAR solution must support the following intents bound to a color:

3. BGP CAR Use-cases

The BGP CAR route may be a transport route or a service route (in this document, we use the term VPN instead of service for simplicity).

3.1. BGP Transport CAR

  • Transport Intent

    • Intent-aware routing between PEs connected across multiple transit domains

      • Set up BGP based end-to-end paths stitching intent-aware intra-domain segments
  • The network diagram below illustrates the reference network topology used in this section for Transport CAR:

            +-----+                +-----+                  +-----+
       .....|S-RR1| <............. |S-RR2| <............... |S-RR3| <...
       :    +-----+                +-----+                  +-----+    :
       :                                                               :
       :                                                               :
       :                                                               :
    +--:--------------+       +-----------------+       +--------------:--+
    |  :              |       |                 |       |              :  |
    |  :              |       |                 |       |              :  |
    |  :          +---|  D=20 |---+         +---|  D=25 |---+          :  |
    |  :          |121|-------|211|         |231|-------|321|          :  |
    |  :          +---| \   / |---+         +---| \   / |---+          :  |
    |----+            |  \ /  |                 |  \ /  |            +----|
    |PE11|            |   V   |                 |   V   |            |PE31|
    |----+            |  / \  |                 |  / \  |            +----|
    |             +---| /   \ |---+         +---| /   \ |---+             |
    |             |122|-------|212|         |232|-------|322|             |
    |             +---|  D=15 |---+         +---|  D=10 |---+             |
    |                 |       |                 |       |                 |
    |    Domain 1     |       |    Domain 2     |       |    Domain 3     |
    +-----------------+       +-----------------+       +------------------+
    
    Figure 3: Transport CAR Reference Topology

    The following network design assumptions apply to the reference topology above, as an example:

    • Independent ISIS/OSPF SR instance in each domain.
    • eBGP peering link between ASBRs (121-211, 121-212, 122-211, 122-212, 231-321, 231-322, 232-321 and 232-322).
    • Peering links have equal cost metric.
    • Peering links have delay configured or measured as shown by "D". D=50 for cross peering links.
    • VPN service is running from PE31 to PE11 via service RRs (S-RRn in figure).
  • The following sections illustrate a few examples of intent use-cases applicable to transport routes.

3.1.1. Use-case of minimization of a cost metric vs a latency metric

  • In the reference topology of Figure 3

    • Each domain has Algo 0 and Flex Algo 128
    • Algo 0 is for minimum cost metric(cost optimized).
    • Flex Algo 128 definition is for minimum delay (low latency).
  • Cost Optimized

    • Color C1 - Minimum cost intent. (Here, a BGP CAR route with Color C1 is being used, instead of BGP-LU.)
    • On PE11, VPN routes colored with C1 are steered via (C1, PE31) BGP CAR route

      • BGP CAR for C1 sets up path(s) between PEs for end-to-end minimum cost.
      • (2) These paths traverse over intra-domain Algo 0 in each domain and account for the peering link cost between ASBRs.
      • Example: PE11 learns (C1, PE31) CAR route via several equal paths:

        1. One such path is through FA0 to node 121, links 121-211, FA0 to 231, link 231-321, FA0 to PE31
        2. Another such path is through FA0 to node 122, link 122-212, FA0 to 232, link 232-322, FA0 to PE31.
  • Minimize latency

    • Color C2 - Minimum latency intent.
    • On PE11, VPN routes colored with C2 are steered via (C2, PE31) BGP CAR route.

      • BGP CAR for C2 advertises paths between PEs for minimum end-to-end delay.
      • (2) These paths traverse over intra-domain Flex Algo 128 in each domain and account for peering link delay between ASBRs.
      • (3) Example: PE11 learns (C2, PE31) BGP CAR route and best path is through FA128 to node 122, link 122-212, FA128 to 232, link 232-322, FA128 to PE31.

3.1.2. Use-case of exclusion/inclusion of link affinity

  • Color C3 - Intent to Minimize cost metric and avoid purple links
  • In the reference topology of Figure 3

    • Each domain has Flex Algo 129 and some links have purple affinity.
    • Flex Algo 129 definition is set to minimum cost metric and avoid purple links (within domain).
    • Peering cross links are colored purple by policy.
  • On PE11, VPN routes colored with C3 are steered via (C3, PE31) BGP CAR route.

    • BGP CAR for C3 sets up paths between PEs for minimum end-to-end cost and avoiding purple link affinity.
    • These paths traverse over intra domain Flex Algo 129 in each domain and accounts for peering link cost between ASBR and avoiding purple links.
    • Example: PE11 learns (C3, PE31) BGP CAR route via 2 paths.

      1. First path is through FA 129 to node 121, link 121-211, FA129 to 231, link 231-321, FA129 to PE31.
      2. Second path is through FA129 to node 122, link 122-212, FA129 to 232, link 232-322, FA129 to PE31.

3.1.3. Use-case of exclusion/inclusion of domains

       +-----+                +-----+                 +-----+
   ....|S-RR1| <............. |S-RR2| <.............. |S-RR3| <....
   :   +-----+                +-----+                 +-----+     :
   :                                                              :
   :                      +----------------+                      :
   :                      |                |                      :
+--:--------------+       |---+        +---|       +--------------:--+
|  :              |   |---|211|        |241|---|   |              :  |
|  :              |   |   |---+        +---|   |   |              :  |
|  :          +---|   |   |    Domain 2    |   |   |---+          :  |
|  :          |121|---|   +----------------+   |---|421|          :  |
|  :          +---|                                |---+          :  |
|----+            |                                |            +----|
|PE11|            |                                |            |PE41|
|----+            |                                |            +----|
|             +---|                                |---+             |
|             |131|---|   +----------------+   |---|431|             |
|             +---|   |   |                |   |   |---+             |
|                 |   |   |---+        +---|   |   |                 |
|    Domain 1     |   |---|311|        |341|---|   |      Domain 4   |
+-----------------+       |---+        +---|       +-----------------+
                          |   Domain 3     |
                          +----------------+
Figure 4

Color C4 - Avoid sending selected traffic via Domain 3

  • VPN routes advertised from PEs with Color C4
  • BGP CAR for Color C4 should only set up paths between PE11 and PE41 that exclude Domain 3

3.1.4. Use-case of virtual network function chains in local and core domains

        ____                      ____
       /    \                    /    \
      | NFV1 |                  | NFV2 |
       \    /                    \    /
+---------------------+  +--------------------+  +-------------------+
|      |E11|          |  |       |E21|        |  |                   |
|      +---+          |  |       +---+        |  |                   |
|                     |  |                    |  |                   |
|                     |  |                    |  |                   |
|----+              +------+                +------+            +----|
|PE11|              | 121  |                | 231  |            |PE31|
|----+              +------+                +------+            +----|
|                     |  |                    |  |                   |
|                     |  |                    |  |                   |
|                     |  |                    |  |                   |
|                     |  |                    |  |                   |
|      Domain 1       |  |      Domain 2      |  |     Domain 3      |
+---------------------+  +--------------------+  +-------------------+
Figure 5
  • Color intent

    • C5 - Routing via min-cost paths
    • C6 - Routing via a local NFV service chain situated at E11
    • C7 - Routing via a centrally located NFV service chain situated at E21
  • Forwarding of packets from PE11 towards PE31:

    • (C5, PE31) mapped packets are sent via nodes 121, 231 to PE31
    • (C6, PE31) mapped packets are sent to E11 and then post-service chain, via 121, 231 to PE31
    • (C7, PE31) mapped packets are sent via 121 to E21 and then post-service chain, via 231 to PE31

3.2. BGP VPN CAR

  • VPN (Service layer) intent

    • Extend the signaling of intent awareness end-to-end: CE site to CE site across provider networks

      • Provide ability for a CE to select paths through specific PEs for a given intent

        • Example-1: Certain intent in transport not available via specific PEs
        • Example-2: Certain CE-PE connection does not support specific intent
        • Example-3: Site access via certain CE does not support specific intent. For instance, link connecting a specific CE to a DC hosting loss-sensitive service may have better quality than a link from another CE
      • Provide ability for a CE to send traffic indicating a specific intent (via suitable encapsulation) to the PE for optimal steering.
    • Intent aware routing support for multiple service (VPN) interworking models

      • Beyond options such as iBGP or Inter-AS Option C that inherently extend from PE to PE

        1. Inter-AS Option A
        2. Inter-AS Option B
        3. GW based interworking(L3VPN, EVPN)
      • Interworking with existing L3VPN deployments, both PEs and CEs
  • The network diagram below illustrates the reference network topology used in this section for VPN CAR.

            +-------------------+           +-------------------+
            |   ....|S-RR|....  |           |  ....|S-RR|.....  |
            |   :   +----+   :  |           |  :   +----+    :  |
            |   :    :  :    :  |           |  :    :  :     :  |
            |----+   :  :   +---|   D=20    |---+   :  :   +----|
           /|PE11|   :  :   |121|-----------|211|   :  :   |PE21|\
     D=25/  |----+   :  :   +---| X       X |---+   :  :   +----|  \ D=25
       /    |        :  :       |   X   X   |       :  :        |    \   V/24
    CE1     |        :  :       |     X D=50|       :  :        |     CE2<---
       X    |        :  :       |   X   X   |       :  :        |    X
    D=15 X  |----+   :  :   +---| X       X |---+   :  :   +----|  X D=10
           X|PE12|...:  :...|212|-----------|232|...:  :...|PE22|X
            |----+          +---|   D=10    |---+          +----|
            |                   |           |                   |
            |     AS 1          |           |     AS 2          |
            +-------------------+           +-------------------+
    
    
    
    Figure 6: VPN CAR reference topology

    The following network design assumptions apply to the reference topology above, as an example:

    • Independent ISIS/OSPF SR instance in each AS.
    • eBGP peering link between VPN ASBRs 121-211, 121-212, 122-211, 122-212.
    • VPN service is running between PEs via service RRs in each AS to local ASBRs. Between ASBRs, its Option-B i.e. next hop self for VPN SAFI.
    • CE1 is dual homed to PE11 and PE12. Similarly, CE2 is dual homed to PE21 and PE22.
    • Peering links have equal cost metric
    • Peering links have delay configured or measured as shown by "D".
    • CE2 advertises prefix V/24 to CE1. It is advertised as RD:V/24 between PEs, including color-awareness
  • The following sections illustrate a few examples of intent use-cases applicable to VPN (service) routes.

3.2.1. Use-case of minimization of a cost metric vs a latency metric

  • In the reference topology of Figure 6

    • Each AS has Flex Algo 0 and 128.
    • Flex Algo 0 is for minimum cost metric(cost optimized).
    • Flex Algo 128 definition is for minimum delay (low latency).
  • Cost Optimized

    • Color C1 - Minimum cost intent.
    • On CE1, flows requiring cost optimized paths to V/24 are steered over (C1, V/24) route.

      • BGP CAR for C1 sets up paths between CEs for minimum end-to-end cost.
      • This advertisement needs BGP CAR between PE-CE for V/24 prefix and color C1 awareness.
      • It also needs BGP VPN CAR between PEs and ASBRs for RD:V/24 prefix and color C1 awareness (C1, RD:V/24).
      • Paths traverse over PE-CE links, intra-domain Flex Algo 0 in each AS and peering links between ASBRs, minimizing cost for VPN.
      • Example: CE1 learns (C1, V/24) CAR route through several equal cost paths:

        1. One path is through link CE1-PE11, FA0 to 121, link 121-211, FA0 to PE21 and link PE21-CE2.
        2. Another such path is through CE1-PE12, FA0 to node 122, link 122-212, FA0 to PE22, link PE22-CE2.
  • Minimize latency

    • Color C2 - Minimum latency intent
    • On CE1, flows requiring low latency paths to prefix V/24 are steered over (C2, V/24) CAR route.

      • BGP CAR for C2 sets up paths between CEs for minimum end-to-end delay.
      • This advertisement needs BGP CAR between PE-CE for V/24 prefix and color C2 awareness.
      • It also needs BGP VPN CAR between PEs and ASBR for RD:V/24 prefix and color C2 awareness (C2, RD:V/24).
      • Paths traverse over intra-domain Flex Algo 128 in each AS and accounts for inter ASBR link delays and PE-CE link delays for the VPN.
      • Example: CE1 learns (C2, V/24) CAR best route through link CE1-PE12, FA128 to 122, link 122-212, FA128 to PE22 and link PE22-CE2.

3.2.2. Use-case of exclusion/inclusion of link affinity

  • Color C3 - Intent to Minimize cost metric and avoid purple links
  • In the reference topology of Figure 6

    • Each AS has Flex Algo 129 and some links have purple affinity.
    • Flex Algo 129 definition is set to minimum cost metric and avoid purple links (within AS).
    • ASBR cross links are colored purple by policy. Bottom PE-CE links are colored purple as well by policy
  • On CE1, flows requiring minimum cost path avoiding purple links to V/24 are steered over (C3, V/24) BGP CAR route.

    • BGP CAR for C3 setup paths between CEs for minimum end-to-end cost and avoiding purple link affinity.
    • This advertisement needs BGP CAR between PE-CE for V/24 prefix and color C3 awareness
    • It also needs BGP VPN CAR between PEs and ASBRs for RD:V/24 prefix and color C3 awareness (C3, RD:V/24).
    • The path avoids purple PE-CE links, traverses over intra-domain Flex Algo 129 in each AS and avoids purple links between VPN ASBRs.
    • Example: CE1 learns (C3, V/24) CAR route through link CE1-PE11, FA129 to 121, link 121-211, FA129 to PE21 and link PE21-CE2.

3.2.3. Use-case of virtual network function chains in local and core domains

                               +-----+
       ........................|S-RR | <.................
       :                       +-----+ <...........     :
       :                                           :    :
       :                                           :    :
       :  ___     ___           ___                :    :
       : /   \   /   \         /   \               :    :
       :| S1  | | S2  |       | S3  |              :    :
       : \   /   \   /         \   /               :    :
     +-:---------------+  +--------------+  +------:----:--+
     | :  |E11|  |E12| |  |    |E21|     |  |      :    :  |
     | :  +---+  +---+ |  |    +---+     |  |      :  +----|<-(V1/24,C1)
     | :               |  |              |  |      :  |PE31|--CE2
     | V               |  |              |  |      :  +----|
     |----+          +------+            +------+  :       |
CE1--|PE11|          | 121  |            | 231  |  :       |
     |----+          +------+            +------+  :  +----|<-(V2/24/C2)
     |                 |  |              |  |      :..|PE32|--CE3
     |                 |  |              |  |         +----|
     |                 |  |              |  |              |
     |                 |  |              |  |              |
     |    Domain 1     |  |   Domain 2   |  |   Domain 3   |
     +-----------------+  +--------------+  +--------------+


Figure 7
  • Color intent

    • C1 - Routing via NFV service chain comprising of [S1, S2] attached to E11 and E12
    • C2 - Routing via NFV service [S3] attached to E21
  • CE1, CE2, CE3 are sites of VPN1.
  • Prefix V1/24 colored with C1 from CE2, and advertised as RD:V1/24 with C1 by PE31 to PE11 via S-RR
  • Prefix V2/24 colored with C2 from CE3, and advertised as RD:V2/24 with C2 by PE32 to PE11 via SS-RR
  • From PE11:

    • [V1/24, C1] mapped packets are sent via S1, S2 and then routed to PE31, CE2
    • [V2/24, C2] mapped packets are sent via S3 and then routed to PE32, CE3

4. Deployment Requirements

The figure below shows a reference large-scale multi-domain network topology for targeted deployments. E1 and E2 are PEs; the other nodes are border routers between domains in different tiers of the network. A VPN route is advertised via service RRs (S-RR) between an egress PE (E2) and an ingress PE (E1). BGP must provide reachability from E1 to E2 based on various intent.

          +-----+                +-----+     V/v via E2   +-----+
   .......|S-RR1|<...............|S-RR2|<.................|S-RR3| <..
   :      +-----+                +-----+     Color C      +-----+   :
   :                                                                :
   :                                                                :
+--:----------+-------------+--------------+-------------+----------:--+
|  :          |             |              |             |          :  |
|  :          |             |              |             |          :  |
|  :        +---+         +---+          +---+         +---+        :  |
|  :        |121|         |231|          |341|         |451|        :  |
|  :        +---+         +---+          +---+         +---+        :  |
|----+        |             |              |             |        +----|
| E1 |        |             |              |             |        | E2 |
|----+        |             |              |             |        +----|
|           +---+         +---+          +---+         +---+           |
|           |122|         |232|          |342|         |452|           |
|           +---+         +---+          +---+         +---+           |
|  Access     |   Metro     |   Core       |   Metro     |   Access    |
|  domain 1   |   domain 2  |   domain 3   |   domain 4  |   domain 5  |
+-------------+-------------+--------------+-------------+-------------+
iPE         iBRM          iBRC           eBRC           eBRM         ePE
Figure 8: Reference large-scale multi-domain network topology

The solution must support the following :

5. Scalability

5.1. Scale Requirements

5.2. Scale Analysis

It is useful to analyze the multiple scaling requirements and specifically the data plane constraints in the context of a few common reference designs and use-cases.

A couple of example scenarios are listed below for reference.

6. Network Availability

7. BGP Protocol Requirements

8. Future Considerations

Multicast service intent

9. Acknowledgements

Many people contributed to this document.

The authors would especially like to thank Jim Uttaro for his guidance on the work and feedback on many aspects of the problem statement. We would also like to thank Daniel Voyer, Luay Jalil and Robert Raszuk for their review and valuable suggestions.

We also express our appreciation to Bruno Decreane, Keyur Patel, Jim Guichard, Alex Bogdanov, Dirk Steinberg, Hannes Gredler and Xiaohu Hu for discussions on several topics that have helped provide input to the document. We also thank Huaimo Chen for his valuable review comments.

The authors would like to thank Stephane Litkowski for his detailed review and for making valuable suggestions to improve the quality of the document. We would also like to thank Kamran Raza and Kris Michelson for their review and comments on the document and to Simon Spraggs, Jose Liste and Jiri Chaloupka for their early inputs on the problem statement.

10. References

10.1. Normative References

[I-D.agrawal-spring-srv6-mpls-interworking]
Agrawal, S., ALI, Z., Filsfils, C., Voyer, D., and Z. Li, "SRv6 and MPLS interworking", Work in Progress, Internet-Draft, draft-agrawal-spring-srv6-mpls-interworking-06, , <https://www.ietf.org/archive/id/draft-agrawal-spring-srv6-mpls-interworking-06.txt>.
[I-D.ietf-bess-srv6-services]
Dawra, G., Filsfils, C., Talaulikar, K., Raszuk, R., Decraene, B., Zhuang, S., and J. Rabadan, "SRv6 BGP based Overlay Services", Work in Progress, Internet-Draft, draft-ietf-bess-srv6-services-08, , <https://www.ietf.org/archive/id/draft-ietf-bess-srv6-services-08.txt>.
[I-D.ietf-idr-bgp-ipv6-rt-constrain]
Patel, K., Raszuk, R., Djernaes, M., Dong, J., and M. Chen, "IPv6 Extensions for Route Target Distribution", Work in Progress, Internet-Draft, draft-ietf-idr-bgp-ipv6-rt-constrain-12, , <https://www.ietf.org/archive/id/draft-ietf-idr-bgp-ipv6-rt-constrain-12.txt>.
[I-D.ietf-idr-tunnel-encaps]
Patel, K., Velde, G. V. D., Sangli, S. R., and J. Scudder, "The BGP Tunnel Encapsulation Attribute", Work in Progress, Internet-Draft, draft-ietf-idr-tunnel-encaps-22, , <https://www.ietf.org/archive/id/draft-ietf-idr-tunnel-encaps-22.txt>.
[I-D.ietf-lsr-flex-algo]
Psenak, P., Hegde, S., Filsfils, C., Talaulikar, K., and A. Gulko, "IGP Flexible Algorithm", Work in Progress, Internet-Draft, draft-ietf-lsr-flex-algo-18, , <https://www.ietf.org/archive/id/draft-ietf-lsr-flex-algo-18.txt>.
[I-D.ietf-spring-segment-routing-policy]
Filsfils, C., Talaulikar, K., Voyer, D., Bogdanov, A., and P. Mattes, "Segment Routing Policy Architecture", Work in Progress, Internet-Draft, draft-ietf-spring-segment-routing-policy-14, , <https://www.ietf.org/archive/id/draft-ietf-spring-segment-routing-policy-14.txt>.
[I-D.ietf-spring-sr-service-programming]
Clad, F., Xu, X., Filsfils, C., Bernier, D., Li, C., Decraene, B., Ma, S., Yadlapalli, C., Henderickx, W., and S. Salsano, "Service Programming with Segment Routing", Work in Progress, Internet-Draft, draft-ietf-spring-sr-service-programming-05, , <https://www.ietf.org/archive/id/draft-ietf-spring-sr-service-programming-05.txt>.
[I-D.ietf-spring-srv6-network-programming]
Filsfils, C., Garvia, P. C., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", Work in Progress, Internet-Draft, draft-ietf-spring-srv6-network-programming-28, , <https://www.ietf.org/archive/id/draft-ietf-spring-srv6-network-programming-28.txt>.
[I-D.voyer-pim-sr-p2mp-policy]
Voyer, D., Filsfils, C., Parekh, R., Bidgoli, H., and Z. Zhang, "Segment Routing Point-to-Multipoint Policy", Work in Progress, Internet-Draft, draft-voyer-pim-sr-p2mp-policy-02, , <https://www.ietf.org/archive/id/draft-voyer-pim-sr-p2mp-policy-02.txt>.
[RFC2119]
Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, , <https://www.rfc-editor.org/info/rfc2119>.
[RFC4360]
Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, , <https://www.rfc-editor.org/info/rfc4360>.
[RFC4684]
Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk, R., Patel, K., and J. Guichard, "Constrained Route Distribution for Border Gateway Protocol/MultiProtocol Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684, , <https://www.rfc-editor.org/info/rfc4684>.
[RFC4760]
Bates, T., Chandra, R., Katz, D., and Y. Rekhter, "Multiprotocol Extensions for BGP-4", RFC 4760, DOI 10.17487/RFC4760, , <https://www.rfc-editor.org/info/rfc4760>.
[RFC5512]
Mohapatra, P. and E. Rosen, "The BGP Encapsulation Subsequent Address Family Identifier (SAFI) and the BGP Tunnel Encapsulation Attribute", RFC 5512, DOI 10.17487/RFC5512, , <https://www.rfc-editor.org/info/rfc5512>.
[RFC5701]
Rekhter, Y., "IPv6 Address Specific BGP Extended Community Attribute", RFC 5701, DOI 10.17487/RFC5701, , <https://www.rfc-editor.org/info/rfc5701>.
[RFC7311]
Mohapatra, P., Fernando, R., Rosen, E., and J. Uttaro, "The Accumulated IGP Metric Attribute for BGP", RFC 7311, DOI 10.17487/RFC7311, , <https://www.rfc-editor.org/info/rfc7311>.
[RFC7606]
Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K. Patel, "Revised Error Handling for BGP UPDATE Messages", RFC 7606, DOI 10.17487/RFC7606, , <https://www.rfc-editor.org/info/rfc7606>.
[RFC8126]
Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, , <https://www.rfc-editor.org/info/rfc8126>.
[RFC8174]
Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, , <https://www.rfc-editor.org/info/rfc8174>.
[RFC8277]
Rosen, E., "Using BGP to Bind MPLS Labels to Address Prefixes", RFC 8277, DOI 10.17487/RFC8277, , <https://www.rfc-editor.org/info/rfc8277>.
[RFC8402]
Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L., Decraene, B., Litkowski, S., and R. Shakir, "Segment Routing Architecture", RFC 8402, DOI 10.17487/RFC8402, , <https://www.rfc-editor.org/info/rfc8402>.
[RFC8664]
Sivabalan, S., Filsfils, C., Tantsura, J., Henderickx, W., and J. Hardwick, "Path Computation Element Communication Protocol (PCEP) Extensions for Segment Routing", RFC 8664, DOI 10.17487/RFC8664, , <https://www.rfc-editor.org/info/rfc8664>.
[RFC8669]
Previdi, S., Filsfils, C., Lindem, A., Ed., Sreekantiah, A., and H. Gredler, "Segment Routing Prefix Segment Identifier Extensions for BGP", RFC 8669, DOI 10.17487/RFC8669, , <https://www.rfc-editor.org/info/rfc8669>.

10.2. Informative References

[I-D.filsfils-spring-sr-policy-considerations]
Filsfils, C., Talaulikar, K., Krol, P., Horneffer, M., and P. Mattes, "SR Policy Implementation and Deployment Considerations", Work in Progress, Internet-Draft, draft-filsfils-spring-sr-policy-considerations-08, , <https://www.ietf.org/archive/id/draft-filsfils-spring-sr-policy-considerations-08.txt>.
[I-D.ietf-idr-performance-routing]
Xu, X., Hegde, S., Talaulikar, K., Boucadair, M., and C. Jacquenet, "Performance-based BGP Routing Mechanism", Work in Progress, Internet-Draft, draft-ietf-idr-performance-routing-03, , <https://www.ietf.org/archive/id/draft-ietf-idr-performance-routing-03.txt>.
[I-D.ietf-mpls-seamless-mpls]
Leymann, N., Decraene, B., Filsfils, C., Konstantynowicz, M., and D. Steinberg, "Seamless MPLS Architecture", Work in Progress, Internet-Draft, draft-ietf-mpls-seamless-mpls-07, , <https://www.ietf.org/archive/id/draft-ietf-mpls-seamless-mpls-07.txt>.
[RFC3906]
Shen, N. and H. Smit, "Calculating Interior Gateway Protocol (IGP) Routes Over Traffic Engineering Tunnels", RFC 3906, DOI 10.17487/RFC3906, , <https://www.rfc-editor.org/info/rfc3906>.
[RFC4271]
Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271, , <https://www.rfc-editor.org/info/rfc4271>.
[RFC4272]
Murphy, S., "BGP Security Vulnerabilities Analysis", RFC 4272, DOI 10.17487/RFC4272, , <https://www.rfc-editor.org/info/rfc4272>.
[RFC4364]
Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, , <https://www.rfc-editor.org/info/rfc4364>.
[RFC6952]
Jethanandani, M., Patel, K., and L. Zheng, "Analysis of BGP, LDP, PCEP, and MSDP Issues According to the Keying and Authentication for Routing Protocols (KARP) Design Guide", RFC 6952, DOI 10.17487/RFC6952, , <https://www.rfc-editor.org/info/rfc6952>.
[RFC7911]
Walton, D., Retana, A., Chen, E., and J. Scudder, "Advertisement of Multiple Paths in BGP", RFC 7911, DOI 10.17487/RFC7911, , <https://www.rfc-editor.org/info/rfc7911>.

Authors' Addresses

Dhananjaya Rao
Cisco Systems
United States of America
Swadesh Agrawal
Cisco Systems
United States of America
Clarence Filsfils
Cisco Systems
Belgium
Ketan Talaulikar
Cisco Systems
India
Bruno Decraene
Orange
France
Dirk Steinberg
Lapishills Consulting Limited
Germany
Luay Jalil
Verizon
United States of America
Jim Guichard
Futurewei
United States of America
Keyur Patel
Arrcus, Inc
United States of America
Wim Henderickx
Nokia
Belgium