Seamless Multicast Interoperability between EVPN and MVPN PEs

4.1. Optimum Forwarding

The solution SHALL support optimum multicast forwarding between EVPN and MVPN PEs within a network (I.e, only one copy will be used to service both EVPN and MVPN PE when same tunnel encapsulation is used). The network can be confined to a CO or it can span across multiple LATAs. The solution SHALL support optimum multicast forwarding with both ingress replication tunnels and P2MP tunnels.¶

4.2. Optimum Replication

For EVPN PEs with IRB capability, the solution SHALL use only a single multicast tunnel among EVPN and MVPN PEs for IP multicast traffic, when both PEs use the same tunnel type. Multicast tunnels can be either ingress replication tunnels or P2MP tunnels. The the solution MUST support optimum replication for both Intra-subnet and Inter-subnet IP multicast traffic:¶

- Non-IP traffic SHALL be forwarded per EVPN baseline [RFC7432] or [RFC8365]¶

- If a Multicast VPN spans across both Intra and Inter subnets, then for Ingress replication regardless of whether the traffic is Intra or Inter subnet, only a single copy of IP multicast traffic SHALL be sent from the source PE to the destination PE.¶

- If a Multicast VPN spans across both Intra and Inter subnets, then for P2MP tunnels regardless of whether the traffic is Intra or Inter subnet, only a single copy of multicast data SHALL be transmitted by the source PE. Source PE can be either EVPN or MVPN PE and receiving PEs can be a mix of EVPN and MVPN PEs - i.e., a multicast VPN can be spread across both EVPN and MVPN PEs.¶

4.3. All-Active and Single-Active Multihoming

The solution MUST support multihoming of source devices and receivers that are sitting in the same subnet (e.g., VLAN) and are multihomed to EVPN PEs. The solution SHALL allow for both Single- Active and All-Active multihoming.¶

4.4. Inter-AS Tree Stitching

The solution SHALL support multicast tree stitching when the tree spans across multiple Autonomous Systems.¶

4.5. EVPN Service Interfaces

The solution MUST support all EVPN service interfaces listed in Section 6 of [RFC7432]:¶

• VLAN-based service interface¶
• VLAN-bundle service interface¶
• VLAN-aware bundle service interface.¶

4.6. Distributed Anycast Gateway

The solution SHALL support distributed anycast gateways as specified in [RFC9135] for tenant workloads on NVE devices operating in EVPN-IRB mode.¶

4.7. Selective & Aggregate Selective Tunnels

The solution SHALL support selective and aggregate selective P- tunnels as well as inclusive and aggregate inclusive P-tunnels. When selective tunnels are used, multicast traffic SHOULD only be forwarded to the remote PEs that have receivers - i.e., if there are no receivers at a remote PE, the multicast traffic SHOULD NOT be forwarded to that PE. If there are no receivers on any remote PEs, then the multicast traffic SHOULD NOT be forwarded to the core.¶

4.8. Tenants' (S,G) or (*,G) states

The solution SHOULD store (C-S,C-G) and (C-*,C-G) states only on PE devices that have an interest in such states hence reducing memory and processing requirements - i.e., PE devices that have sources and/or receivers interested in such multicast groups.¶

4.9. Zero Disruption upon BD/Subnet Addition

In DC environments, various broadcast domains (BDs) are provisioned and removed on a regular basis due to host mobility, policy, and tenant changes. Such change in BD configuration SHOULD NOT affect existing flows within the same BD or any other BD in the network.¶

4.10. No Changes to Existing EVPN Service Interface Models

VLAN-aware bundle service as defined in Section 6.3 of [RFC7432] typically does not require any VLAN ID translation from one tenant site to another - i.e., the same set of VLAN IDs are configured consistently on all tenant segments. In such scenarios, EVPN-IRB multicast service MUST maintain the same mode of operation and SHALL NOT require any VLAN ID translation.¶

4.11. External source and receivers

The solution SHALL support sources and receivers external to the tenant domain. i.e., multicast source inside the tenant domain can have receiver outside the tenant domain and vice versa.¶

4.12. Tenant RP placement

The solution SHALL support a tenant to have RP anywhere in the network. RP can be placed inside the EVPN network or MVPN network or external domain.¶

5.1. IRB Unicast versus IRB Multicast

[RFC9135] describes the operation for EVPN PEs in IRB mode for unicast traffic. The same IRB model is used for unicast traffic, where an IP-VRF in an EVPN PE is attached to one or more bridge tables (BTs) via virtual IRB interfaces, is also applicable for multicast traffic.¶

For unicast traffic, the intra-subnet traffic is bridged within the MAC-VRF associated with that subnet (i.e., a lookup based on MAC Destination Address (MAC-DA) is performed); whereas, the inter-subnet traffic is routed in the corresponding IP-VRF (i.e. a lookup based on IP Destination Address (IP-DA) is performed).¶

A given tenant can have one or more IP-VRFs; however, without loss of generality, this document assumes one IP-VRF per tenant. In context of a given tenant's multicast traffic, the intra-subnet traffic is bridged for non-IP traffic and it is Layer-2 switched for IP traffic. Whereas, the tenant's inter-subnet multicast traffic is always routed in the corresponding IP-VRF. The difference between bridging and L2-switching for multicast traffic is that the former uses MAC-DA lookup for forwarding the multicast traffic; whereas, the latter uses IP-DA lookup for such forwarding where the forwarding states are built in the MAC-VRF using IGMP/MLD or PIM snooping.¶

5.2. Operational Model for EVPN IRB PEs

Without the loss of generality, this section assumes that all EVPN PEs have IRB capability and operating in IRB mode for both unicast and multicast traffic (e.g., all EVPN PEs are homogenous in terms of their capabilities and operational modes). As it will be seen later, an EVPN network can consist of a mix of PEs where some are capable of multicast IRB and some are not and the multicast operation of such heterogeneous EVPN network will be an extension of an EVPN homogenous network. Therefore, we start with the multicast IRB solution description for the EVPN homogenous network.¶

The EVPN PEs terminate IGMP/MLD messages from tenant host devices or PIM messages from tenant routers on their IRB interfaces, thus avoid sending these messages over MPLS/IP core. A tenant virtual/physical router (e.g., CE) attached to an EVPN PE becomes a multicast routing the adjacency of that PE. Furthermore, the PE uses MVPN BGP protocol and procedures per [RFC6513] and [RFC6514]. With respect to multicast routing protocol between the tenant's virtual/physical router and the PE that it is attached to, any of the following PIM protocols is supported per [RFC6513]: PIM-SM with Any Source Multicast (ASM) mode, PIM-SM with Source Specific Multicast (SSM) mode, and PIM Bidirectional (BIDIR) mode. Support of PIM-DM (Dense Mode) is excluded in this document per [RFC6513].¶

The EVPN PEs use MVPN BGP routes defined in [RFC6514] to convey tenant (S,G) or (*,G) states to other MVPN or EVPN PEs and to set up overlay trees (inclusive or selective) for a given MVPN instance. The root or a leaf of such an overlay tree is terminated on an EVPN or MVPN PE. Furthermore, this inclusive or selective overlay tree is terminated on a single IP-VRF of the EVPN or MVPN PE. In case of EVPN PE, these overlay trees never get terminated on MAC-VRFs of that PE.¶

Overlay trees are instantiated by underlay provider tunnels (P- tunnels) - e.g., P2MP, MP2MP, or unicast tunnels per [RFC6513]. When there are several overlay trees mapped to a single underlay P-tunnel, the tunnel is referred to as an aggregate tunnel.¶

Figure-1 below depicts a scenario where a tenant's multicast VPN spans across both EVPN and MVPN PEs; where all EVPN PEs have multicast IRB capability. An EVPN PE (with multicast IRB capability) can be modeled as an MVPN PE where the virtual IRB interface of an EVPN PE (virtual interface between a BT and IP-VRF) can be considered a routed interface for the MVPN PE.¶

                     EVPN PE1
                  +------------+
        Src1 +----|(MAC-VRF1)  |                   MVPN PE3
       Rcvr1 +----|      \     |    +---------+   +--------+
                  |    (IP-VRF)|----|         |---|(IP-VRF)|--- Rcvr5
                  |      /     |    |         |   +--------+
        Rcvr2 +---|(MAC-VRF2)  |    |         |
                  +------------+    |         |
                                    |  MPLS/  |
                     EVPN PE2       |  IP     |
                  +------------+    |         |
        Rcvr3 +---|(MAC-VRF1)  |    |         |    MVPN PE4
                  |       \    |    |         |   +--------+
                  |    (IP-VRF)|----|         |---|(IP-VRF)|--- Rcvr6
                  |       /    |    +---------+   +--------+
        Rcvr4 +---|(MAC-VRF3)  |
                  +------------+

Figure 2 depicts the modeling of EVPN PEs based on MVPN PEs where an EVPN PE can be modeled as a PE that consists of an MVPN PE whose routed interfaces (e.g., attachment circuits) are replaced with IRB interfaces connecting each IP-VRF of the MVPN PE to a set of BTs. Similar to an MVPN PE where an attachment circuit serves as a routed multicast interface for an IP-VRF associated with an MVPN instance, an IRB interface serves as a routed multicast interface for the IP-VRF associated with the MVPN instance. Since EVPN PEs run MVPN protocols (e.g., [RFC6513] and [RFC6514] ), for all practical purposes, they look just like MVPN PEs to other PE devices. Such modeling of EVPN PEs transforms the multicast VPN operation of EVPN PEs to that of MVPN and thus simplifies the interoperability between EVPN and MVPN PEs to that of running a single unified solution based on MVPN.¶

                     EVPN PE1
                  +------------+
        Src1 +----|(MAC-VRF1)  |
                  |     \      |
       Rcvr1 +----|  +--------+|    +---------+   +--------+
                  |  |MVPN PE1||----|         |---|MVPN PE3|--- Rcvr5
                  |  +--------+|    |         |   +--------+
                  |      /     |    |         |
        Rcvr2 +---|(MAC-VRF2)  |    |         |
                  +------------+    |         |
                                    |  MPLS/  |
                     EVPN PE2       |  IP     |
                  +------------+    |         |
        Rcvr3 +---|(MAC-VRF1)  |    |         |
                  |       \    |    |         |
                  |  +--------+|    |         |   +--------+
                  |  |MVPN PE2||----|         |---|MVPN PE4|--- Rcvr6
                  |  +--------+|    |         |   +--------+
                  |       /    |    +---------+
        Rcvr4 +---|(MAC-VRF3)  |
                  +------------+

Although modeling an EVPN PE as a MVPN PE, conceptually simplifies the operation to that of a solution based on MVPN, the following operational aspects of EVPN need to be factored in when considering seamless integration between EVPN and MVPN PEs.¶

• Unicast route advertisements for IP multicast source¶
• multihoming of IP multicast sources and receivers¶
• Mobility for Tenant's sources and receivers¶

5.3. Unicast Route Advertisements for IP multicast Source

When an IP multicast source is attached to an EVPN PE, the unicast route for that IP multicast source needs to be advertised. When the the source is attached to a Single-Active multihomed Ethernet Segment (ES), then the EVPN DF PE is the PE that advertises a unicast route corresponding to the source IP address with VRF Route Import extended community which in turn is used as the Route Target for Join (S,G) messages sent toward the source PE by the remote PEs. The EVPN PE advertises this unicast route using EVPN route type 2 and VPN-IP unicast route along with VRF Route Import extended community. EVPN route type 2 is advertised with the Route Targets corresponding to both IP-VRF and MAC-VRF/BT; whereas, the VPN-IP unicast route is advertised with RT corresponding to the IP-VRF. When unicast routes are advertised by MVPN PEs, they are advertised using VPN-IP unicast route along with VRF Route Import extended community per [RFC6514].¶

When the multicast source is attached to an All-Active multihomed ES, then the PE that learns the source advertises the unicast route for that source using EVPN route type 2 and VPN-IP unicast route along with VRF Route Import extended community. EVPN route type 2 is advertised with the Route Targets corresponding to both IP-VRF and MAC-VRF/BT; whereas, the VPN-IP unicast route is advertised with RT corresponding to the IP-VRF. When the other multihoming EVPN PEs for that ES receive this unicast EVPN route, they import the route and check to see if they have learned the route locally for that ES, if they have, then they do nothing. But if they have not, then they add the IP and MAC addresses to their IP-VRF and MAC-VRF/BT tables respectively with the local interface corresponding to that ES as the corresponding route adjacency. Furthermore, these PEs advertise an VPN-IP unicast route along with VRF Route Import extended community and Route Target corresponding to IP-VRF to other remote PEs for that MVPN. Therefore, the remote PEs learn the unicast route corresponding to the source from all multihoming PEs associated with that All-Active ES even though one of the multihoming PEs may only have directly learned the IP address of the source.¶

EVPN PEs advertise unicast routes as host routes using EVPN route type 2 for sources that are directly attached to a tenant BD that has been extended in the EVPN fabric. EVPN PE may summarize sources (IP networks) behind a router that is attached to itself or sources that are connected to a BD, which is not extended across EVPN fabric and advertises those routes with EVPN route type 5(i.e., IP Prefix route as per Section 3 of [RFC9136]). EVPN host routes are also advertised as VPN-IP host routes to MVPN PEs only in case of seamless interop mode.¶

Section 8 extends seamless interop procedures to EVPN only fabrics as an IRB solution for multicast. L3VPN provisioning is not needed among EVPN PEs. EVPN PEs only need to advertise unicast routes using EVPN route type 2 or route type 5 with VRF Route Import extended community and don't need to advertise VPN-IP routes within EVPN only fabric.¶

Section 9 discusses Data Center Interconnect (DCI) use cases, where EVPN and MVPN networks are connected using a gateway model. In the gateway model, EVPN PE advertises unicast routes as VPN-IP routes along with VRI extended community for all multicast sources are attached behind EVPN PEs. All VPN-IP routes SHOULD be summarized while adverting to MVPN PEs.¶

5.4. Multihoming of IP Multicast Source and Receivers

EVPN [RFC7432] has extensive multihoming capabilities that allow Tenant Systems (TSs) to be multihomed to two or more EVPN PEs in Single-Active or All-Active mode. In Single-Active mode, only one of the multihoming EVPN PEs can receive/transmit traffic for a given subnet (a given BD) for that multihomed Ethernet Segment (ES). In All-Active mode, any of the multihoming EVPN PEs can receive/transmit unicast traffic but only one of them (the DF PE) can send BUM traffic to the multihomed ES for a given subnet.¶

The multihoming mode (Single-Active versus All-Active) of a TS source can impact the MVPN procedures as described below.¶

5.5. Mobility for Tenant's Sources and Receivers

When a tenant system (TS), source or receiver, is multihomed behind a group of multihoming EVPN PEs, then TS mobility is supported among EVPN PEs. Furthermore, such TS mobility will cause an temporary disruption to the related multicast service among EVPN and MVPN PEs. If a source is moved from one EVPN PE to another PE, then the EVPN mobility procedure discovers this move and a new unicast route advertisement (using both EVPN and VPN-IP routes) is made by the EVPN PE where the source has moved to per Section 5.3 above and unicast route withdrawal (for both EVPN and VPN-IP routes) is performed by the EVPN PE where the source has moved from.¶

The move of a source results in disruption of the IP multicast flow for the corresponding (S,G) flow till the new unicast route associated with the source is advertised by the new PE along with the VRF Route Import EC, the join messages sent by the egress PEs are received by the new PE, the multicast state for that flow is installed in the new PE and a new overlay tree is built for that source from the new PE to the egress PEs that are interested in receiving that IP multicast flow.¶

The move of a receiver results in disruption of the IP multicast flow to that receiver only till the new PE for that receiver discovers the source and joins the overlay tree for that flow.¶

6.1. Intra-ES Subnet Tunnel

As described in Section 5.4.2, when a multicast source is sitting behind an All-Active ES, then an intra-subnet multicast tunnel is needed among the multihoming EVPN PEs for that ES to carry multicast flow received by one of the multihoming PEs to the other PEs in that ES. We refer to this multicast tunnel as an Intra-ES subnet tunnel. It is recommended to use Ingress replication for Intra-ES subnet tunnel(especially for dual-homing usecases), In case of multihoming to three or more EVPN PEs, then other tunnel types such as P2MP, MP2MP, BIER, and Assisted Replication can be considered. It should be noted that this intra-ES subnet tunnel is only needed for All-Active multihoming and it is not required for Single-Active multihoming.¶

The EVPN PEs belonging to a given All-Active ES discover each other using EVPN Ethernet Segment route per procedures described in [RFC7432] Section 8. These EVPN PEs perform DF election per [RFC7432], [RFC8584], or other DF election algorithms to decide who is a DF for a given BD. If the BD belongs to a tenant that has IRB IP multicast enabled for it, then for fixed-mode, each PE sets up an intra-ES subnet tunnel to forward IP multicast traffic received locally on that BD to other multihoming PE(s) for that ES. Therefore, IP multicast traffic received via a local attachment circuit is sent on this tunnel and on the associated IRB interface for that BT and other local attachment circuits if there are interested receivers for them. The other multihoming EVPN PEs treat this intra-ES subnet tunnel just like their local ACs - i.e., the multicast traffic received over this tunnel is treated as if it is received via its local AC. Thus, the multihoming PEs cannot receive the same IP multicast flow from an MVPN tunnel (e.g., over an IRB interface for that BD) because between a source behind a local AC versus a source behind a remote PE, the PE always chooses its local AC.¶

When all multihomed PE support [RFC9251], traffic may be forwarded on demand basis. Based on IGMP/MLD synchronization procedure specified in [I-D.ietf-bess-evpn-igmp-mld-proxy], the join state may be synchronized between all multihomed PEs. Multihomed PE which receives the multicast traffic from its attached circuit, may send the traffic towards intra-ES subnet tunnel, only if it has received an IGMP/MLD sync message from one of the multihomed PEs. Such extension is outside the scope of this document and may be covered in a separate document if required.¶

In case of a TS, receiver sits behind an All-Active Multihoming ES and a TS source sits behind an inter-subnet tunnel (with respect to the multihomed PE), it is possible that more than one multihomed PEs sends MVPN join toward remote PE based on incoming join on their local interfaces. When the traffic is received on the inter-subnet tunnel, it is sent towards locally attached receivers. Only DF sends traffic towards multihomed ethernet segment. Traffic received on the inter-subnet tunnel, should not be sent towards Intra-ES subnet tunnel.¶

When ingress replication is used for intra-ES subnet tunnel, every PE in the All-Active multihoming ES has all the information to setup these tunnels - i.e., a) each PE knows what are the other multi- homing PEs for that ES via EVPN Ethernet Segment route and can use this information to setup intra-ES subnet tunnel among themselves.¶

6.2. Intra-Subnet BUM Tunnel

As the name implies, this tunnel is setup to carry BUM traffic for a given subnet/BD among EVPN PEs. In [RFC7432], this overlay tunnel is used for transmission of all BUM traffic including tenant IP multicast traffic.¶

When an EVPN IRB PE operates in seamless interop mode, this tunnel is used for all broadcast, unknown-unicast, non-IP multicast traffic, and link-local IP multicast traffic - i.e., it is used for all BUM traffic except for tenant IP multicast traffic. This tunnel is setup using the IMET route for a given EVI/BD. The composition and advertisement of IMET routes are exactly per [RFC7432]. It should be noted that when an EVPN All-Active multihoming PE uses both this tunnel as well as intra-ES subnet tunnel, there SHALL be no duplication of multicast traffic over the network because they carry different types of multicast traffic - i.e., intra-ES subnet tunnel among multihoming PEs carries only tenant IP multicast traffic; whereas, intra-subnet BUM tunnel carries link-local IP multicast traffic and BUM traffic (w/ non-IP multicast).¶

6.3. Inter-Subnet IP Multicast Tunnel

As its name implies, this tunnel is setup to carry IP-only multicast traffic for a given tenant across all its subnets (BDs) among EVPN and MVPN PEs.¶

The following NLRIs from [RFC6514] is used for setting up this inter- subnet tunnel in the network.¶

Intra-AS I-PMSI A-D route is used for the setup of default underlay tunnel (also called inclusive tunnel) for a tenant IP- VRF. The tunnel attributes are indicated using the PMSI attribute with this route.¶

S-PMSI A-D route is used for the setup of Customer flow specific underlay tunnels. This enables selective delivery of data to PEs having active receivers and optimizing fabric bandwidth utilization. The tunnel attributes are indicated using PMSI attribute with this route.¶

Each EVPN PE supporting a specific MVPN instance discovers the set of other PEs in its AS that are attached to sites of that MVPN using Intra-AS I-PMSI A-D route (route type 1) per [RFC6514]. It can also discover the set of other ASes that have PEs attached to sites of that MVPN using Inter-AS I-PMSI A-D route (route type 2) per [RFC6514]. After the discovery of PEs that are attached to sites of the MVPN, an inclusive overlay tree (I-PMSI) can be setup for carrying tenant multicast flows for that MVPN; however, this is not a requirement per [RFC6514] and it is possible to adopt a policy in which all tenant flows are carried on S-PMSIs.¶

An EVPN-IRB PE sends a tenant IP multicast flow to other EVPN and MVPN PEs over this inter-subnet tunnel that is instantiated using MVPN I-PMSI or S-PMSI. This tunnel can be considered as being originated and terminated from/to among IP-VRFs of EVPN/MVPN PEs; whereas, intra-subnet tunnel originated/terminated among MAC-VRFs of EVPN PEs.¶

6.4. IGMP/MLD Hosts as TSes

IGMP/MLD messages are terminated by the EVPN-IRB PE and tenant (*,G) or (S,G) join messages are sent via MVPN Shared Tree Join route (route type 6) or Source Tree Join route (route type 7) respectively of MCAST-VPN NLRI per [RFC6514].¶

Here, IGMP/MLD states are terminated at IRB interfaces, and local interest are expressed in the context of IP-VRF to remote PEs.¶

In the case of a network with only IGMP/MLD hosts, the preferred mode of operation is that of Shortest Path Tree(SPT) per Section 14 of [RFC6514]. This mode is only supported for PIM-SM and avoids the RP configuration overhead. Such mode is chosen by provisioning/ configuration.¶

6.5. PIM Routers as TSes

Just like an MVPN PE, an EVPN PE runs a separate tenant multicast routing instance (VPN-specific) per MVPN instance and the following tenant multicast routing instances are supported:¶

• • PIM Sparse Mode (PIM-SM) with the ASM service model¶
  • PIM Sparse Mode with the SSM service model¶
  • PIM Bidirectional Mode (BIDIR-PIM), which uses bidirectional tenant-trees to support the ASM service model¶

A given tenant's PIM join messages for (*,G) or (S, G) are processed by the corresponding tenant multicast routing protocol and they are advertised over MPLS/IP network using the Shared Tree Join route (route type 6) and Source Tree Join route (route type 7) respectively of MCAST-VPN NLRI per [RFC6514].¶

7.1. Intra-Subnet L2 Switching

Rcvr1 in Figure 1 is connected to PE1 in MAC-VRF1 (same as Src1) and sends IGMP/MLD join for (C-S, C-G), IGMP/MLD snooping will record this state in the local bridging entry. A routing entry will be formed as well which will point to MAC-VRF1 as RPF for Src1. We assume that Src1 is known via ARP or similar procedures. Rcvr1 will get a locally bridged copy of multicast traffic from Src1. Rcvr3 is also connected in MAC-VRF1 but to PE2 and hence would send IGMP/MLD join which will be recorded at PE2. PE2 will also form routing entry and RPF will be assumed as Tenant Tunnel "Tenant1" formed beforehand using MVPN procedures. Also, this would cause the multicast control plane to initiate a BGP MCAST-VPN type 7 route which would include VRI for PE1 and hence be accepted on PE1. PE1 will include Tenant1 tunnel as Outgoing Interface (OIF) in the routing entry. Now, since it has knowledge of remote receivers via MVPN control plane it will encapsulate original multicast traffic in Tenant1 tunnel towards core.¶

7.2. Inter-Subnet L3 Routing

Rcvr2 in Figure 1 is connected to PE1 in MAC-VRF2 and hence PE1 will record its membership in MAC-VRF2. Since MAC-VRF2 is enabled with IRB, gets added as another OIF to the routing entry formed for (C-S, C-G). Rcvr2 and Rcvr4 are also in different MAC-VRFs than multicast speaker Src1 and hence need Inter-subnet forwarding. PE2 now adds another OIF 'MAC-VRF2' to its existing routing entry. But there is no change in control plane states since it is already sent MVPN route and no further signaling is required. Traffic received by the tenant tunnel interface gets routed towards both MAC-VRF1 and MAC-VRF3. PE3 forms routing entry very similar to PE2. It is to be noted that PE3 does not have MAC-VRF1 configured locally but still can receive the multicast data traffic over the Tenant1 tunnel formed due to MVPN procedures and routes traffic towards its L3 OIFs for that (C-S,C-G).¶

8.1. Setup of overlay multicast delivery

It must be emphasized that this solution poses no restriction on the setup of the tenant BDs and that neither the source PE, nor the receiver PEs do not need to know/learn about the BD configuration on other PEs in the tenant IP-VRF ( Since EVPN PE is modeled as MVPN PE, source and receivers are announced to remote PE in the context of tenant IP-VRF(MVPN) as opposed to BD context). The Reverse Path Forwarder (RPF) is selected per the tenant multicast source and the IP-VRF in compliance with the procedures in [RFC6514], using the incoming EVPN route type 2 or 5 NLRI per [RFC7432].¶

The VRF Route Import (VRI) extended community that is carried with the VPN-IP routes in [RFC6514] MUST be carried with the EVPN unicast routes when these routes are used. The construction and processing of the VRI are consistent with [RFC6514]. The VRI MUST uniquely identify the PE which is advertising a multicast source and the IP- VRF it resides in.¶

VRI may be constructed as following:

• The 4-octet Global Administrator field can be set to an IP address of the PE ( PE's loopback address or VTEP address) or BGP router-id.¶
• The 2-octet Local Administrator can be set to a number that uniquely identifies the IP-VRF within the PE.¶

EVPN PE MUST have Route Target Extended Community to import/export MVPN routes. In a data center environment, it is desirable to have this RT is configured using an auto-generated method rather than a static configuration.¶

The following is one recommended model to auto-generate MVPN RT:

• The Global Administrator field of the MVPN RT MAY be set to BGP AS Number used for EVPN/MVPN session.¶
• The Local Administrator field of the MVPN RT MAY be set to the VNI associated with the tenant IP-VRF.¶

Every PE that detects a local receiver via a local IGMP/MLD join or a local PIM join for a specific source (overlay SSM mode) MUST terminate the IGMP/PIM signaling at the IP-VRF and generate a (C-S,C- G) via the BGP MCAST-VPN route type 7 per [RFC6514] if and only if the RPF for the source points to the fabric. If the RPF points to a local multicast source on the same MAC-VRF or a different MAC-VRF on that PE, the MCAST-VPN MUST NOT be advertised and data traffic will be locally routed/bridged to the receiver.¶

The VRI received with EVPN route type 2 or 5 NLRI from source PE will be appended as an export route-target extended community. The PE which has advertised the unicast route with VRI, will import the incoming MCAST-VPN NLRI in the IP-VRF with the same import route- target extended-community and other PEs SHOULD ignore it. Following such procedure the source PE learns about the existence of at least one remote receiver in the tenant overlay and programs data plane accordingly, so that a single copy of multicast data is forwarded into the fabric using tenant VRF tunnel(i.e. inter-subnet tunnel/mvpn tunnel).¶

If the multicast source is unknown (overlay ASM mode), the MCAST-VPN route type 6 (C-*,C-G) join SHOULD be targeted towards the designated overlay Rendezvous Point (RP) by appending the received RP VRI as an export route-target extended community. Every PE which detects a a local source, registers with its RP PE. That is how the RP learns about the tenant source(s) and group(s) within the MVPN. Once the overlay RP PE receives either the first remote (C-RP,C-G) join or a local IGMP/MLD/PIM join, it will trigger an MCAST-VPN route type 7 (C- S,C-G) towards the actual source PE for which it has received PIM register messages in full compliance with regular PIM procedures. This involves the source PE to advertise the MCAST-VPN Source Active A-D route (MCAST-VPN route-type 5) towards all PEs. The Source Active A-D route is used to inform all PEs in a given MVPN about the active multicast source for switching from RPT to SPT when MVPNs use tenant RP-shared trees (i.e., rooted at tenant's RP) per Section 13 of [RFC6514].¶

8.2. Handling of different encapsulations

Just as in [RFC6514] the MVPN I-PMSI and S-PMSI A-D routes are used to form the overlay multicast tunnels and signal the tunnel type using the P-Multicast Service Interface Tunnel (PMSI Tunnel) attribute.¶

9.1. Control plane inter-connect

The gateway(s) MUST be setup with the inclusive set of all the IP- VRFs that span across the two domains. On each gateway, there will be at least two BGP sessions: one towards the DC side and the other towards the WAN side. Usually for redundancy purposes, more sessions are setup on each side. The unicast route propagation follows the exact same procedures in [RFC9014]. Hence, a multicast host located in either domain, is advertised with the gateway IP address as the next-hop to the other domain. As a result, PEs view the hosts in the other domain as directly attached to the gateway and all inter-domain multicast signaling is directed towards the gateway(s). Received MVPN routes type 1-7 from either side of the gateway(s), MUST NOT be reflected back to the same side but processed locally and re- advertised (if needed) to the other side:¶

• Intra-AS/Inter-AS I-PMSI A-D Route: these are distributed within each domain to form the overlay tunnels which terminate at gateway(s). They are not passed to the other side of the gateway(s).¶

• C-Multicast Route: joins are imported into the corresponding IP- VRF on each gateway and advertised as a new route to the other side with the following modifications (the rest of NLRI fields and path attributes remain on-touched):¶

  • Route-Distinguisher is set to that of the IP-VRF¶
  • Route-target is set to the exported route-target list on IP-VRF¶
  • The PMSI tunnel attribute and BGP Encapsulation extended community will be modified according to Section 8¶
  • Next-hop will be set to the IP address which represents the gateway on either domain¶
• Source Active A-D Route: same as joins¶
• S-PMSI A-D Route: these are passed to the other side to form selective PMSI tunnels per every (C-S,C-G) from the gateway to the PEs in the other domain provided it contains receivers for the given (C-S, C-G). Similar modifications made to joins are made to the newly originated S-PMSI.¶

In addition, the Originating Router's IP address is set to GW's IP address. Multicast signaling from/to hosts on local ACs on the gateway(s) are generated and propagated in both domains (if needed) per the procedures in section 6 in this document and in [RFC6514] with no change. It must be noted that for a locally attached source, the gateway will program an OIF per every domain from which it receives a remote join in its forwarding plane and different encapsulation will be used on the data packets.¶

9.2. Data plane inter-connect

Traffic forwarding procedures on gateways are the same as those described for PEs in Section 5 except that, unlike a non-border leaf PE, the gateway will not only route the incoming traffic from one side to its local receivers, but will also send it to the remote receivers in the other domain after de-capsulation and appending the right encapsulation. The OIF and IIF are programmed in FIB based on the received joins from either side and the RPF calculation to the source or RP. The de-capsulation and encapsulation actions are programmed based on the received I-PMSI or S-PMSI A-D routes from either side.¶

The multicast traffic from local sources on each gateway may flow to the other gateway with either of the tunnel encapsulation. But, it is recommended to use VxLAN tunnel than MPLS in this case.¶

10.1. Interaction with L2EVPN PE and Seamless interop capable PE

The following cases are considered in this section.¶

• Case1: [RFC9251] is supported both at seamless interop capable PE and L2EVPN PE.¶
• Case2: [RFC9251] is supported only at seamless interop capable PE.¶
• Case3: [RFC9251] is not supported at interop capable PE.¶

[RFC9251] support is recommended for seamless interop capable PE. SEMG can group L2 EVPN PEs into two separate groups ( one that supports the [RFC9251] and another that doesn't) from IMET routes that it receives from the remote peers. The interop procedure for handling these two different sets of remote L2 EVPN PEs are captured in case 1 and 2.¶

Case 1: [RFC9251] is supported both at seamless interop capable PE and L2EVPN PE¶

This may be the most common usecase.¶

SEMG-DF has the following special responsibilities on a BD for which it is the DF.¶

• Process EVPN SMET routes from the remote L2 EVPN PEs that support [RFC9251] and creates L2 multicast state. SMET route in-turn triggers the creation of L3 multicast state similar to the IGMP/MLD join received on the local AC. SEMG-DF exercises the MVPN procedures for the join.¶
• It should not process IGMP/MLD control packets from L2EVPN PE that supports [RFC9251].¶
• Originate SMET(*,*) route towards L2 EVPN PEs. This is to receive traffic from multicast sources that are connected behind L2 EVPN PEs.¶

• When SEMG-DF receives traffic from L2 EVPN PE on the intra-subnet tunnel on BD-X, it does the following¶

  • Performs FHR functionality¶
  • Advertises the host route with L3 label and VRF Route-Import corresponds to the tenant domain.¶
  • Sends the traffic towards the locally attached receivers.¶
  • Sends the traffic towards L2EVPN receiver on BDs other than incoming BD(after multicast routing)¶
  • Sends the traffic towards remote seamless interop capable PEs, where receivers are attached/connected behind that PE.¶

• When SEMG-DF receives traffic from the MVPN tunnel, it does the following¶

  • Sends the traffic toward the IRB interfaces, where the receiver exists¶
  • BD corresponding to the IRB interfaces may have local receivers or remote receivers behind L2 EVPN PE. SEMG-DF sends the traffic on the intra-subnet tunnel for remote receivers.¶

Case 2: [RFC9251] is not supported at L2 EVPN PE¶

This case only differs from case 1 in terms of the way it learns receivers behind L2 EVPN PEs and how SEMG-DF attracts traffic from sources behind L2 EVPN PE. The rest of the procedures specified above is applicable for this case.¶

SEMG-DF has the following special responsibilities on a BD for which it is the DF¶

• Process IGMP/MLD control packets from remote L2 EVPN PEs that doesn't support [RFC9251] and create L2 and L3 state.¶
• When an IGMP/MLD query is received on the intra-subnet tunnel on BD-X, SEMG-DF needs to send proxy IGMP/MLD reports for all groups that it has learned from remote L2-EVPN PEs on that BD.¶
• Connecting multicast router behind L2 EVPN PE is not recommended. If a multicast router is connected behind L2 EVPN PE, the BD corresponds to the VRF tunnel needs to be configured in the L2 EVPN PE so that the PIM router may get all joins that are received in the BD corresponds to the MVPN tunnel interface at SEMG-DF.¶
• SEMG-DF should get all multicast traffic from L2EVPN PEs. This may be achieved by sending an IGMP/MLD query or PIM hello on the intra- subnet tunnel¶

Case 3: [RFC9251] is not supported at seamless interop capable PE¶

The procedure for handling this use case is exactly the same as the case 2.¶

All seamless interop capable PEs other than SEMG should discard SMET routes that are coming from L2EVPN PEs and must discard all IGMP/MLD control packets, if any received on the intra-subnet tunnel. SEMG should discard incoming SMET routes and IGMP/MLD joins from L2EVPN PEs, if it is not the DF for the incoming BD.¶

When [RFC9251] is supported both at seamless interop capable PE and L2EVPN PE, selective forwarding is done based on receiver interest at the egress-PE, when overlay tunnel type is Ingress-replication or selective tunnel.¶

10.2. Network having L2EVPN PE, Seamless interop capable PE and MVPN PE

Since MVPN PE can only interact with Seamless interop capable PEs, SEMG-DF acts as FHR and LHR for sources and receivers behind L2 EVPN PE. Only SEMG-DF advertises the VPN-IP unicast route along with the VRF Route Import extended community for hosts behind L2 EVPN PE. No additional procedures are required when they all co-exist.¶

A.1. Policy based model

Policies may be used to enforce EVPN BUM procedure for traffic flows with TTL value 1. Traffic flow that matches the policy is excluded from seamless interop procedure specified in this document, hence TTL decrement issue will not apply.¶

A.2. Exercising BUM procedure for VLAN/BD

Servers/hosts sending the traffic with TTL value 1 may be attached to a separate VLAN/BD, where multicast routing is disabled. When multicast routing is disabled, EVPN BUM procedure may be applied to all traffic ingressing on that VLAN/BD. On the Egress PE, the RPF for such traffic may be set to BD interface, where the source is attached.¶

A.3. Intra-subnet bridging

The procedure specified in the section enables a PE to detect an attached subnet source (i.e., source that is directly attached in the tenant BD/VLAN). By applying the following procedure for the attached source, Traffic flows having TTL value 1 can be supported.¶

• • On the ingress PE, do the bridging on the interface towards the core interface¶
  • On the egress side, make a decision whether to bridge or route at the outgoing interface (OIF) based on whether the source is attached to the OIF's BD/VLAN or not.¶

Recent ASIC supports single lookup forwarding for bridging and routing (L2+L3). The procedure mentioned here leverages this ASIC capability.¶

                    PE1
                   +------------+
           S11 +---+(BD1)       |  +---------+
                   |  \         |  |         |
                   |(IP-VRF)-(CORE)|         |
                   |  /         |  |         |
           R12 +---+(BD2)       |  |         |
                   +------------+  |         |
                                   |         |
                    PE2            | VXLAN.  |
                   +------------+  |         |
           R21 +---+(BD1)       |  |         |
                   |  \         |  |         |
                   |(IP-VRF)-(CORE)|         |
                   |  /         |  |         |
           R22+----+(BD3)       |  +---------+
                   +------------+

Consider the above picture. In the picture

     - PE1 and PE2 are seamless interop capable PEs
     - S11 is a multicast host directly attached to PE1 in BD1
     - Source S11 sends traffic to Group G11
     - R21, R22 are IGMP/MLD receivers for group G11
     - R21 and R22 are attached to BD1 and BD3 respectively at PE2.

When source S11 starts sending the traffic, PE1 learns the source and announces the source using MVPN procedures to the remote PEs.¶

At PE2, IGMP/MLD joins from R21, R22 result the creation of (*,G11) entry with outgoing OIF as IRB interface of BD1 and BD3. When PE2 learns the source information from PE1, it installs the route (S11, G11) at the tenant VRF with RPF as the CORE interface.¶

PE2 inherits (*, G11) OIFs to (S11, G11) entry. While inheriting OIF, PE2 checks whether the source is attached to OIF's subnet. OIF matching source subnet is added with a flag indicating bridge only interface. In the case of (S11, G11) entry, BD1 is added as the bridge only OIF, while BD3 is added as normal OIF(L3 OIF). PEs (PE2) sends MVPN join (S11, G11) towards PE1, since it has local receivers.¶

At Ingress PE(PE1), CORE interface is added to (S11, G11) entry as an OIF (outgoing interface) with a flag indicating that bridge only interface. With this procedure, ingress PE(PE1) bridges the traffic on the CORE interface. (PE1 retains the TTL and source-MAC). The traffic is encapsulated with VNI associated with the CORE interface. PE1 also routes the traffic for R12 which is attached to BD2 on the same device.¶

PE2 decapsulates the traffic from PE1 and does an inner lookup on the tenant VRF associated with incoming VNI. Traffic lookup on the tenant VRF yields (S11, G11) entry as the matching entry. Traffic gets bridged on BD1 (PE2 retains the TTL and source-MAC) since the OIF is marked as a bridge only interface. Traffic gets routed on BD2.¶