Current Options for Securing Global Routing

4.1.1. Integrity Verification and Authentication

MD5 protection of the TCP session header, described in [RFC5925], was the first available mechanism to protect the integrity of a BGP session. It has been obsoleted by the TCP Authentication Option (TCP-AO; [RFC5925]), which offers stronger protection. While MD5 is still the most used mechanism due to its availability in vendors' equipment, TCP-AO SHOULD be preferred when implemented by both sides of a session.¶

Optionally, if TCP-AO is not supported, while both sides of the BGP session can support a stronger authentication algorithm than MD5, such as SHA-1 or SHA-256, using the stronger method SHOULD be considered. Aside from that, using keychain-based cryptographic keys lifecycle management, as suggested in [RFC6518] is highly RECOMMENDED.¶

Additionally, IPsec could also be used for session protection. At the time of publication, there has been no wide-spread adoption of using IPsec for BGP sessions, and further analysis is required to define guidelines.¶

The drawback of TCP session protection is additional configuration and management overhead for the maintenance of authentication information (for example, MD5 passwords). In either case, protection of TCP sessions used by BGP SHOULD be enabled when BGP sessions are established over shared networks where the risk of spoofing is high (like IXPs). Operators are also RECOMMENDED to consider the trade-offs and apply BGP session protection on all other external BGP sessions as well.¶

Aside of this, most vendors use simple, reverse-decryptable password hash algorithm to store shared secrets keys for BGP (and other routing protocols) in devices' configuration files. While this practice simplifies password management tasks, since the passwords can always easily be deciphered, it carries the risk of leaking this information if a configuration is shared, e.g., with a vendor for a support case, or if the device is decommissioned and later resold without having been wiped. Hence, if a device offers more secure storage mechanisms for secrets, these SHOULD be used.¶

Furthermore, operators SHOULD block spoofed packets (packets with a source IP address not belonging to their IP address space) at all edges of their network (see [RFC2827] and [RFC3704] ). This protects the TCP session used by Internal BGP (iBGP) from attackers outside the Autonomous System. Similarly, the considerations for using non globally reachable addresses for links handling BGP sessions from Section 3.1 apply accordingly.¶

Furthermore, as an additional security measure, iBGP sessions SHOULD also be protected using the authentication mechanisms discussed above.¶

4.1.2. Defending Against PMTUD Related Attacks

In 2018 an attack on BGP was described in the literature which claims to enable BGP route injection without Layer 2 adjacency by leveraging PMTUD, see ([FENG-22]). The attack leverages packet fragmentation to bypass standard TCP protection mechanisms, so routes can be injected into an established BGP session. While the attack would be mitigated by the integrity mechanisms suggested in Section 4.1.1, operators SHOULD additionally take precautions to defend against these attacks, especially if authentication mechanisms are not in use. To mitigate this attack, BGP speakers should not allow packet fragmentation on the control plane for BGP traffic between themselves and their neighbors. This is feasible, as even on multi-hop sessions, the path MTU should be known to the operators, meaning that it can be statically and consistently configured for both speakers involved in a session to prevent the need for fragmentation. Hence, operators SHOULD ensure that fragmentation is neither allowed nor necessary for BGP packets between two BGP speakers. If this is not possible, a strict lower limit for the MTU SHOULD be configured. This is usually done for TCP packets like those for a BGP session using MSS (Maximum Segment Size) clamping. Given that IPv6 requires an MTU of at least 1280b [RFC8200], and to keep clamping consistent between IPv4 and IPv6, an MTU of 1280b, i.e., an MSS of 1240b for IPv4 and 1220b for IPv6, is the RECOMMENDED minimum.¶

5.1.1. IPv4 Special-Purpose Prefixes

The IANA IPv4 Special-Purpose Address Registry [IANAv4Spec] maintains the list of IPv4 special-purpose prefixes and their routing scope, and it SHOULD be used for prefix-filter configuration. Prefixes with value "False" in column "Global" SHOULD be discarded on Internet BGP sessions (eBGP).¶

5.1.2. IPv6 Special-Purpose Prefixes

The IANA IPv6 Special-Purpose Address Registry [IANAv6Spec] maintains the list of IPv6 special-purpose prefixes and their routing scope, and it SHOULD be used for prefix-filter configuration. Only prefixes with value "False" in column "Global" SHOULD be discarded on Internet BGP sessions.¶

5.2.1. IANA-Allocated Prefix Filters

IANA has allocated all the IPv4 available space. Therefore, there is no reason why operators would keep checking that prefixes they receive from BGP neighbors are in the IANA-allocated IPv4 address space [IANAv4Reg]. No specific filters need to be put in place by operators who want to make sure that IPv4 prefixes they receive in BGP updates have been allocated by IANA.¶

For IPv6, given the size of the address space, it can be seen as wise to accept only prefixes derived from those allocated by IANA. Operators can dynamically build this list from the IANA- allocated IPv6 space [IANAv6Reg]. As IANA keeps allocating prefixes to RIRs, the aforementioned list should be checked regularly against changes, and if they occur, prefix filters should be computed and pushed on network devices. The list could also be pulled directly by routers when they implement such mechanisms. As there is delay between the time an RIR receives a new prefix and the moment it starts allocating portions of it to its LIRs, there is no need for doing this step quickly and frequently. However, operators SHOULD ensure that all IPv6 prefix filters are updated within a maximum of one month after any change in the list of IPv6 prefixes allocated by IANA.¶

If the process in place (whether manual or automatic) cannot guarantee that the list is updated regularly, then it's better not to configure any filters based on allocated networks. The IPv4 experience has shown that many network operators implemented filters for prefixes not allocated by IANA but did not update them on a regular basis. This created problems for the latest allocations, and required extra work for RIRs that had to "de-bogonize" the newly allocated prefixes. (See [RIPE-351] for information on de-bogonizing.)¶

5.4.1. IPv4

Typically, the 0.0.0.0/0 prefix is not intended to be accepted or advertised except in specific customer/provider configurations; general filtering outside of these is RECOMMENDED.¶

5.4.2. IPv6

Typically, the ::/0 prefix is not intended to be accepted or advertised except in specific customer/provider configurations; general filtering outside of these is RECOMMENDED.¶

6.1.1. Route Objects

The corner stone of IRR based information are ROUTE (IPv4) and ROUTE6 (IPv6) objects. These document, for a given prefix, the AS/ASes allowed to originate the prefix. Note that for a given prefix also more specific objects may exist. However, technically, the semantic of a ROUTE/ROUTE6 object is that of an exact match.¶

Operators SHOULD create ROUTE/ROUTE6 objects for all prefixes they do or do plan to originate.¶

6.1.2. AS-SETs

An AS-SET is an object that contains AS numbers or other AS-SETs. The purpose of AS-SETs is creating a recursively queryable structure documenting the cone of an AS. An operator may create an AS-SET defining all AS numbers of its customers. A transit provider might create an AS-SET listing the AS numbers or AS-SETS of those ASes it provides upstream to. In turn, these ASes describe the AS numbers/AS-SETS of their customers, etc. Using recursion, it is possible to retrieve from an AS-SET the complete list of AS numbers that the neighbor is likely to announce. For each of these AS numbers, it is also easy to look in the corresponding IRR for all associated prefixes.¶

Please note that different IRR may provide conflicting data, especially on AS-SETs. Recently, an attack was observed where a malicious party created an empty AS-SET for a large transit provider (see [NLNOG-22]). As it was created in an RIR database often taking precedent over other IRR sources, several ASes imported this empty AS-SET, and hence filtered all prefixes advertised by this transit provider. To mitigate this issue, hierarchical AS-SETs reside in the IRR of the RIR and explicitly list the ASN to which they pertain, e.g., AS65536:AS-EXAMPLE. Additionally, the IRR source may also be referenced: RIPE::AS65536:AS-EXAMPLE.¶

Operators SHOULD create a hierarchical AS-SET representing their cone. If AS-SETs are included in another AS-SET, they SHOULD be hierarchical.¶

6.1.3. Recursively Computing Filters

Using AS-SETs and ROUTE/ROUTE6 objects, it is possible to use the IRR information to build, for a given neighbor AS, a list of prefixes the neighbor is authorized to originated or transited. This can be done relatively easily using scripts and existing tools capable of retrieving this information from the registries. This approach is exactly the same for both IPv4 and IPv6.¶

The macro-algorithm for the script is as follows. For the neighbor that is considered, the distant operator has provided the AS and may be able to provide a hierarchically named AS-SET object (aka AS-MACRO). With these two mechanisms, a script can build, for a given neighbor, that lists allowed prefixes and the AS number from which they should be originated. One could decide not to use the origin information and only build monolithic prefix filters from fetched data combining prefixes a neighbor is authorized to transit and originate.¶

As prefixes, AS numbers, and AS-SETs may not all be under the same RIR authority, it is difficult to choose for each object the appropriate IRR to poll. Some IRRs have been created and are not restricted to a given region or authoritative RIR. They allow RIRs to publish information contained in their IRR in a common place. They also make it possible for any subscriber (probably under contract) to publish information too. When doing requests inside such an IRR, it is possible to specify the source of information in order to have the most reliable data. One could check a popular IRR containing many sources (such as RADb [RADb], the Routing Assets Database) and only select as sources some desired RIRs and trusted major ISPs (Internet Service Providers).¶

As objects in IRRs may frequently vary over time, it is important that prefix filters computed using this mechanism are refreshed regularly. Refreshing the filters on a daily basis SHOULD be considered because routing changes must sometimes be done in an emergency and registries may be updated at the very last moment. Note that this approach significantly increases the complexity of the router configurations, as it can quickly add tens of thousands of configuration lines for some important neighbors, e.g., large peers or downstreams. To manage this complexity, operators could use, for example, bgpq4 [bgpq4], a set of tools making it possible to simplify the creation of automated filter configuration from policies stored in an IRR.¶

6.2.1. Route Origin Validation (ROV)

If route origin validation is implemented, the reader SHOULD refer to the rules described in [RFC7115]. In short, each external route received on a router SHOULD be checked against the Resource Public Key Infrastructure (RPKI) data set:¶

• If a corresponding ROA (Route Origin Authorization) is found and is valid, then the prefix SHOULD be accepted.¶
• If the ROA is found and is INVALID, then the prefix SHOULD be discarded.¶
• If a ROA is not found, then the prefix SHOULD be accepted, but the corresponding route SHOULD be given a low preference.¶

In addition to this, operators SHOULD sign their routing objects so their routes can be validated by other networks running origin validation. Please note that, when signing routing objects, operators SHOULD strive to create minimally covering ROAs for their intended announcements, see [RFC7115] and [RFC9319], to reduce the attack surface of forged-origin hijacks and attempts to exhaust routers' route processing capacity in terms of memory and CPU [KIRIN-22]. For example, if an operator received a /29 allocation and intends to announce it in a deaggregation of /32, the corresponding ROA should cover the /29 with a longest allowed prefix of /32, instead of signing for a deaggregation up until /48.¶

One should understand that the RPKI model brings new, interesting challenges. The paper "On the Risk of Misbehaving RPKI Authorities" [hotRPKI] explains how the RPKI model can impact the Internet if authorities don't behave as they are supposed to. Further analysis is certainly required on RPKI, which carries part of BGP security.¶

6.2.2. Autonomous System Provider Authorization (ASPA)

If autonomous system provider authorization is implemented, the reader SHOULD refer to the rules described in [I-D.ietf-sidrops-aspa-verification]. In short, each external route received on a router SHOULD be checked against the ASPA record found in the Resource Public Key Infrastructure (RPKI) based on the relationship to the neighbor.¶

In [I-D.ietf-sidrops-aspa-verification], see following sections based on the neighbor relationship:¶

• Section 6.1 for routes received from customer, lateral peer, by an RS from an RS-client, or by an RS-client from an RS.¶
• Section 6.2 for routes received from an upstream or mutual-transit neighbor.¶

ASPA validation can result in one of three outcomes, VALID, INVALID, and UNKNOWN.¶

• If a route's AS_PATH is evaluated as VALID, then the prefix SHOULD be accepted.¶
• If a route's AS_PATH is evaluated as INVALID, then the prefix SHOULD be discarded, and the event logged.¶
• If a route's AS_PATH is evaluated as UNKNOWN, then the prefix SHOULD be accepted, and the event logged. The corresponding route MAY be given a low preference.¶

6.5.1. Outbound Filtering Prefixes Using BGP Roles

To ensure that no prefixes leak via AS relationships (routes learned from peers or upstreams to other peers or upstreams), [RFC9234] introduces BGP roles and the BGP Only to Customer (OTC) attribute. The OTC attribute forms a tandem with ASPA, see Section 6.2.2. Operators SHOULD configure appropriate roles according to Section 3 of [RFC9234] to enable prefix filtering based on BGP relationships. Furthermore, for prefixes imported from upstreams, the OTC attribute SHOULD be set and evaluated according to [RFC9234], Section 5:¶

6.5.2. Outbound Filtering Using Large-Communities

Despite a fall-back mechanism being implemented to support one-sided BGP roles, they must be supported by both neighbors in a BGP session to be fully effective. To completely cover an AS, all neighbors should utilize BGP roles on their sessions. Hence, if at least one neighbor does not yet utilize BGP roles, or if the operator cannot deploy BGP roles and/or use the OTC attribute on their own infrastructure, operators SHOULD additionally utilize BGP large-communities to annotate where they learned prefixes and filter accordingly on sessions where they re-announce these prefixes, see [RFC8195]. While technically possible, standard BGP communities (see [RFC1997]) SHOULD NOT be used for this purpose due to the prevalence of 32bit ASNs which can only be represented in large-communities (see [RFC8092]).¶

Operators SHOULD designate a large community namespace for each neighbor relationship, for example, OPERATOR_ASN:100:NEIGHBOR_ASN for upstreams, OPERATOR_ASN:101:NEIGHBOR_ASN for peers, OPERATOR_ASN:102:NEIGHBOR_ASN for downstreams, etc. These communities SHOULD cover all relationships documented in Section 3 of [RFC9234]. Additionally, if operators allow downstreams to announce more specifics than generally accepted in the GRT (see [CCR-22]), they should dedicate a large-community list to that purpose as well, to ensure they can effectively prevent re-announcements of these prefixes.¶

For information on how these annotations SHOULD be included in filter sets, please see Section 8.¶

6.6.1. IXP LAN Prefix Filtering

Within the IXP community, most IXPs prefer the IXP LAN prefix to not be advertised to the GRT ([TBD]). While some IXPs may opt to advertise the IXP LAN prefix, e.g., with the route server's ASN, operators present on an IXP MUST respect the choice of the IXP regarding the advertisement state of the IXP LAN prefix. Furthermore, e.g., the RIPE region now reached consensus on reducing the initial IXP allocation size for IPv4 (see [RIPE-804]) above their own limits on maximum prefix lengths acceptable in the GRT (see [RIPE-399] and [RIPE-532]). When a network is present on an IXP and has sessions with other IXP members over a common subnet (IXP LAN prefix), it SHOULD NOT accept exact matches or more-specific prefixes for the IXP LAN prefix from any of its external BGP neighbors. Accepting these routes may create a black hole for connectivity to the IXP LAN. To reduce the risk of accidental route leaks of IXP LAN prefixes for which the corresponding IXP opted to not have them in the GRT, operators MAY choose to use "BGP next-hop-self" on all routes learned on that IXP to not be required to distribute the IXP LAN Prefix within their IGP. Furthermore, IXPs may opt to create ROAs indicating AS0 as the only valid origin AS if they want to prevent their prefixes from being announced on the Internet.¶

If the IXP LAN prefix is accepted at all, it SHOULD only be accepted from the ASes that the IXP authorizes to announce it -- this will usually be automatically achieved by filtering announcements using RPKI and/or IRR database.¶

6.6.2. Prefixes on Routers Connected to an IXP

It is suggested (see also [APNICTRN-17]), that operators dedicate routers for connections to an IXP that SHOULD only carry routes from the ASes cone, and not a full-table or default-route. This reduces the chance of accidental route leaks and prevents other IXP members from pointing default routes via the IXP LAN to such a router. Alternative, MAY use a separate routing context (e.g. VRF) for IXP peerings, which only containins routes form the local AS cone.¶

6.6.3. PMTUD and the Loose uRPF Problem

Originally, in order to have PMTUD working in the presence of loose uRPF, it would be necessary that all the networks that may source traffic that could flow through the IXP have a route for the IXP LAN prefix. This relates to "packet too big" ICMP messages sent by IXP members' routers potentially being sourced using an address of the IXP LAN prefix. In the presence of loose uRPF, this ICMP packet is dropped if there is no route for the IXP LAN prefix or a less specific route covering the IXP LAN prefix.¶

Hence, similar to considerations in Section 3 regarding non globally routable transit networks, IXP members SHOULD ensure that "packet too big" ICMP messages sent by their routers have a source address in IP address space advertised to the GRT, e.g., the router's loopback address. Note that this issue causes service interruption in case of lost "packet too big" messages, but may also reduce debuggability in, e.g., traceroutes. If they decide to implement this behavior for all ICMP messages, operators SHOULD ensure that this address is only used for ICMP messages egressing via the interface connected to the IXP LAN. Otherwise, readability of traceroutes will be significantly reduced, as the specific interface a packet passed through is no longer visible in traceroutes.¶

7.2.1. Maximum Prefixes on a Single Session

It is RECOMMENDED to configure a limit on the number of routes to be accepted from a neighbor. The following rules are generally RECOMMENDED:¶

• For peers, it is RECOMMENDED to have a limit lower than the number of routes in the Internet. This will shut down the BGP session if the neighbor suddenly advertises the full table. Operators can also configure different limits for each neighbor to which they have a peering relationshipt, according to the number of routes they are supposed to advertise, plus some headroom to permit growth. However, please note that these limits may change over time. Hence, it is RECOMMENDED that both neighbors clearly communicate the number of prefixes they expect to announce to each other and agree on a way to automatically update this information in the future. At the time of writing, ([PeeringDB]) is a common source for programmatically obtaining suggested prefix limits for neighbors. In the absence of communicated prefix limits, the number of expected prefixes can be inferred from the AS-SET, see Section 6.1.2. It is RECOMMENDED to include additional headroom of 20% when utilizing an inferred prefix limit.¶
• From upstreams that provide full routing, it is RECOMMENDED to have a limit higher than the number of routes in the Internet. A limit is still useful in order to protect the network (and in particular, the routers' memory) if too many routes are sent by the upstream. The limit should be chosen according to the number of routes that can actually be handled by routers.¶

It is important to regularly review the limits that are configured as the Internet can quickly change over time. Some vendors propose mechanisms to have two thresholds: while the higher number specified will shut down the session, the first threshold will only trigger a log and can be used to passively adjust limits based on observations made on the network.¶

7.2.2. Maximum Prefixes per Neighboring AS

With RPKI allowing operators to sign ROAs specifying a minimum and maximum prefix length (contrary to ROUTE/ROUTE6 objects), researchers noted that this allows deaggregation attacks ([KIRIN-22]). By configuring a ROA that cover an, e.g., /32, one can effectively authorize an AS to announce 65536 unique prefixes. Leveraging the by now large availability of free and/or cheap opportunities to obtain IPv6 upstream, a malicious party could leverage this to cause significant Internet wide route churn and GRT growth. By constantly advertising and withdrawing prefixes, churn exceeding the size of the IPv6 fulltable at the time of writing (around 200k prefixes) could be created by constantly announcing and withdrawing prefixes to upstream ASes at various PoPs.¶

It is therefore RECOMMENDED that operators, in addition to per-session prefix limits, implement a global limit to the number of prefixes they accept per neighboring AS. As with the per-session limits, these SHOULD be sensible and regularly updated in coordination with operators from neighboring ASes.¶

7.3.1. AS_PATH Filtering

This section lists the RECOMMENDED practices when processing BGP AS_PATHs in addition the considerations from Section 6.¶

• Operators SHOULD should follow Section 6.1 to only accept 2-byte or 4-byte AS_PATHs from customers containing ASNs belonging to (or authorized to transit through) the customer. If operators cannot build and generate filtering expressions to implement this, they SHOULD consider accepting only path lengths relevant to the type of customer they have (as in, if these customers are a leaf or have customers of their own) and SHOULD try to discourage excessive prepending in such paths. This loose policy SHOULD be combined with filters for specific 2-byte or 4-byte AS_PATHs that must not be accepted if advertised by the customer, such as upstream transit providers or peer ASNs.¶
• Operators SHOULD NOT accept prefixes with private AS numbers in the AS_PATH unless the prefixes are from customers. In any case, operators SHOULD NOT re-export prefixes with AS_PATHs containing private AS numbers. An exception could occur when an upstream is offering some particular service like black-hole origination based on a private AS number: in that case, prefixes SHOULD be accepted. Customers should be informed by their upstream in order to put in place ad hoc policy to use such services.¶
• Operators SHOULD NOT accept prefixes when the first AS number in the AS_PATH is not the one of the neighbor unless the BGP session is setup towards a BGP route server [RFC7947] (for example, on an IXP) with transparent AS_PATH handling. In that case, this verification needs to be deactivated, as the first AS number will be the one of an IXP member, whereas the neighbor's AS number will be the one of the BGP route server.¶
• Operators SHOULD NOT advertise prefixes with a nonempty AS_PATH unless they are either announcing prefixes from the GRT to downstreams, or if the whole AS_PATH is within their cone.¶
• Operators SHOULD NOT advertise prefixes with upstream AS numbers in the AS_PATH to any AS except to those to which they provide upstream transit.¶
• Private AS numbers are conventionally used in contexts that are "private" and SHOULD NOT be used in advertisements to BGP neighbors that are not party to such private arrangements, and they SHOULD be stripped when received from BGP neighbors that are not party to such private arrangements. Additionally, operators MAY decide to not accept prefixes with private AS numbers in their AS_PATH at all.¶
• Operators SHOULD NOT accept their own AS number in the AS_PATH by overriding BGP's default behavior. When considering an exception, the impact (which may be severe on routing) should be evaluated carefully.¶
• Overly long AS_PATH, i.e., longer than 64 entries, may cause issues for some older routing hardware. Hence, operators SHOULD NOT use excessive prepending when advertising prefixes. Excessive prepending is defined as any prepending that leads to an AS_PATH exceeding 64 entries across the GRT. Additionally, it is RECOMMENDED that operators filter any prefix advertised with an AS_PATH of more than 64 entries.¶

7.3.2. AS_PATH Manipulation

This section lists the RECOMMENDED practices when manipulating BGP AS_PATHs, to limit chances of accidentally producing AS_PATHs that would have to be filtered by neighbors according to Section 7.3.1.¶

Some BGP implementations offer various advanced AS_PATH manipulation features, such as overriding or rewriting a part of the AS_PATH. For instance, a very commonly used mechanism is the so-called "AS Override" feature, primarily intended for use in MPLS L3 VPNs, where the customer's AS number is overridden with the provider's AS number, to allow site-to-site communication where both customer sites use the same AS number. Some vendors went even further, offering a possibility to fully rewrite or even delete the AS_PATH Attribute from incoming or outgoing BGP Update messages.¶

Furthermore, AS_PATH filtering is an option when ASN renumbering is done. Such an operation is common, and mechanisms exist to allow smooth ASN migration [RFC7705]. The usual migration technique, local to a router, consists of modifying the AS_PATH so it is presented to a neighbor with the previous ASN, as if no renumbering was done. This makes it possible to change the ASN of a router without reconfiguring all eBGP neighbors at the same time (as that operation would require synchronization with all neighbors attached to that router). During this renumbering operation, the rules described above may be adjusted.¶

In principle, use of any AS_PATH modification mechanism except AS_PATH prepend in the public Internet SHOULD be avoided at all. Also, as discussed already, AS_PATH prepends SHOULD NOT be excessive. Operators are RECOMMENDED to not prepend more than five times. The "AS Override" feature MAY still be used in closed environments, such as VPNs not directly exchanging any NLRIs with the Internet. AS_PATH rewriting/deleting SHOULD be avoided. Especially the practice of providing upstream to customers using a private ASN and then using rewriting on either side is strongly NOT RECOMMENDED.¶

7.5.1. Inbound BGP Community Scrubbing

• Operators SHOULD scrub inbound BGP communities with their ASN in the high-order bits, unless they have been documented and communicated to neighbors to be used as a signaling mechanism. If a received NLRI contains an excessive amount of BGP communities, i.e., more than 100, operators MAY truncate the list of BGP communities. When truncating BGP communities, operators SHOULD prioritize retaining BGP communities of their neighbors.¶
• Extended BGP communities (see [RFC4360]) received from external neighbors SHOULD be scrubbed. However, there are operational circumstances where it MAY be reasonable to accept extended BGP communities from neighbors, see, e.g., [RFC4364], Inter-AS VPN Option B.¶
• When known, BGP communities used to signal RPKI ROV state (see Section 6.2.1) received from eBGP neighbors MUST be scrubbed. This is done to prevent BGP update storms in case a neighbor looses the connection to its validator, changing the validation state of previously valid NLRI, and thereby the applied community for those NLRI, triggering an update message. For further details on why BGP communities MUST NOT be used to signal RPKI ROV state, please see [I-D.spaghetti-sidrops-avoid-rpki-state-in-bgp].¶

7.5.2. Outbound BGP Community Scrubbing

• Operators SHOULD scrub outbound BGP communities with their ASN in the high-order bits, unless their semantics have been documented and communicated to neighbors. Even if the semantics of BGP communities has been documented, operators SHOULD be mindful of the number of BGP communities they add to NLRI. When sending NLRI to neighbors, operators SHOULD limit the number of BGP communities they communicate to the outside, i.e., not overload NLRI with an excessive amount of BGP communities by outbound filtering all non-externally useful BGP communities they added for use within their own network. Furthermore, when defining BGP communities, operators MUST be careful not to define redundant BGP communities or using multiple BGP communities to express properties that could sensibly be represented with a single community.¶
• Extended BGP communities (see [RFC4360]) SHOULD NOT be sent to eBGP neighbors. However, there are operational circumstances where it MAY be reasonable to send extended BGP communities to neighbors, see, e.g., [RFC4364], Inter-AS VPN Option B.¶
• Operators MUST NOT use BGP communities to signal RPKI ROV state, see [I-D.spaghetti-sidrops-avoid-rpki-state-in-bgp]. If an operator is still in a migration phase discontinuing this practice, they MUST scrub any such community they use for signaling RPKI-ROV state before sending NLRI to their external neighbors. This is done to prevent the propagation of BGP update storms in case one of their BGP speakers looses the connection to its validator, changing the validation state of previously valid NLRI, and thereby the applied community for those NLRI, triggering an update message. For further details on why BGP communities MUST NOT be used to signal RPKI ROV state, please see [I-D.spaghetti-sidrops-avoid-rpki-state-in-bgp].¶

7.6.1. BGP Attribute Scrubbing

Over the past years several instances of network disruptions due to routers being unable to process specific BGP attributes were encountered. As such, operators MAY opt to temporarily scrub specific BGP attributes known to cause service disruptions on their infrastructure. Operators SHOULD NOT scrub unknown transitive attributes in general.¶

However, while being a very useful tool, BGP attribute scrubbing features may cause undesired effects and sometimes even large-scale outages as well. Therefore, they MUST NOT be used as a permanent solution, but only as a last-resort temporary workaround. Furthermore, removing mandatory BGP attributes and optional attributes commonly used in the Internet, such as AS_PATH, Communities, MED etc. may have a significant negative impact beyond an operator's own AS. Hence, it is RECOMMENDED that such attributes are never removed when importing NLRI.¶

When sending NLRI to external neighbors, operators SHOULD avoid sending not yet standardized or only internally used attributes, i.e., scrub attributes they added which are not in public use before exporting NLRI.¶

7.6.2. BGP Attribute Header Correction

BGP attributes are stored within BGP UPDATE messages as a vector of Type-Length-Value (TLV) fields. The Attribute Type field contains a set of control bits, such as the Optional Bit (set to 1 for Optional Attributes and 0 for Well-Known), the Transitive Bit (specifying whether the attribute is Transitive, i.e., should be propagated outside the local AS or, Non-Transitive, i.e., should not be propagated outside the local AS) etc. Initially, [RFC4271] mandated that a BGP speaker tears down a BGP session when receiving even a single UPDATE message containing a malformed combination of Attribute TLV headers. However, [RFC7606] allows BGP implementers to optionally add features providing self-correction of malformed attributes in a limited number of cases.¶

Operators MAY use such, self-correcting mechanisms for BGP Attribute TLV headers. However, they SHOULD consider the operational impact such features have, SHOULD monitor for cases where such self-correction is necessary, and SHOULD follow up on such cases to ensure that root-causes are identified and addressed.¶

7.8.1. Not Setting a Higher LOCAL_PREF for NLRI received via an IXP

Given that traffic forwarded via an IXP can be more cost-efficient than sending that same traffic via an upstream, many operators set a higher LOCAL_PREF for NLRI received via an IXP. This means that all traffic from the AS and all members of its cone routing via this AS will preferentially be routed via these paths (see [RFC4271]), effectively overriding the effect of AS_PATH prepending, see also [I-D.ietf-grow-as-path-prepending].¶

As noted in [I-D.ietf-grow-as-path-prepending], setting a higher LOCAL_PREF on IXP links means that neighbors on the IXP can no longer use AS_PATH prepending for, e.g., traffic engineering. More crucially, it prevents operators from draining traffic flowing via an IXP when necessary, e.g., prior to a scheduled maintenance. Especially when NLRI are exchanged via an RS, simply terminating the session is usually not possible without also impacting other neighbors.¶

Hence, operators SHOULD NOT set a higher local preference for NLRI received via an IXP RS. Instead, other non-transitive methods, e.g., setting a corresponding MED on imported routes, should be preferred.¶

If, when trying to drain traffic on an IXP link via AS_PATH prepending of NLRI sent to the RS, an operator encounters an IXP member ignoring these prepends, they may be able to selectively widthdraw routes from being announced to that member by using communities documented by the IXP to prevent the RS from exporting their NLRI to that specific IXP member.¶

7.8.2. Honoring GSHUT on an IXP

Graceful BGP Session Shutdown (GSHUT) as defined in [RFC8326] is a formalized method for draining traffic from sessions gracefully before, e.g., maintenance. However, while AS_PATH prepending does not have to be supported by two neighbors, GSHUT requires all neighbors to implement it by implementing a policy that assigns a lower LOCAL_PREF to NLRI matching the GRACEFUL_SHUTDOWN BGP community.¶

GSHUT is a more effective method of traffic draining than, e.g., AS_PATH prepending. Hence, in general, GSHUT SHOULD be supported on all eBGP sessions. However, as an IXP member, when ignoring the previous recommendation and setting a higher LOCAL_PREF for sessions via an IXP LAN, GSHUT MUST be supported.¶

8.1.1. Implicit Policies and Default Behavior

Almost all BGP implementations have specific default behavior, including behavior when reaching the end of a policy, behavior when no policy is defined (even though [RFC8212] now requires a default-deny in the absence of policy), etc. However, default behavior and matching characteristics may differ between vendors and implementations. Implicitly relying on vendor-specific default behavior can pose issues if a network operator migrates from one vendor to the other, or when operating a mixed-vendor environment. Furthermore, implicit defaults may change, requiring intervention by operators. Therefore, it is RECOMMENDED that operators create explicit policy statements, even for behavior covered by defaults. Such a practice helps simplifying automation of router configurations, and prevents incidents due to changing or differing implicit defaults, especially when migrating between vendors and in interoperability scenarios.¶

8.1.2. Order of Prefixfilters

BGP is a policy-based routing protocol with import/export policies controlling advertisements/acceptance of NLRI (see [RFC4272] Sec. 9.1), and BGP sessions without policy being applied should default to a deny-all stance (see [RFC8212]). The specific implementation of import/export policies varies between vendors in terms of complexity and naming, from basic prefix-based / AS_PATH-based filters, to complex IF-THEN-like policy structures (typical names are: "route maps", "route policies", "policy statements").¶

Independent of the implementation, all BGP policies consist of one or more rule sets, that are executed in a sequence, one after another. The first rule set will scan the complete content of Adj-RIBs-in or Adj-RIBs-out; NLRIs permitted by a rule set will be passed to subsequent rule sets, while denied prefixes are discarded.¶

Policies SHOULD avoid computationally expensive setups, or setups of rules that apply computation to NLRI that will subsequently be discarded. Hence, the more prefixes a rule is likely to discard, the earlier it SHOULD be evaluated.¶

To further illustrate this, you can find an (incomplete) example for a simple inbound filter for a session with a neighbor in a peer relationship who has 60 prefixes in its cone creating additional load. We assume that, accidentally, an IPv4 fulltable of 1,000,000 entries is being sent. 2,500 NLRI contain unregistered/private AS numbers, 500 NLRI relate to bogon prefixes, and 5,000 NLRI are RPKI invalid, with none of the routes in the neigbor's cone falling in any of these categories. The number of operations per line are given in parentheses.¶

• 1. Add community imported from peer (1,000,000)¶
• 2. Add community imported in LOCATION (1,000,000)¶
• 3. Reject NLRI with private AS numbers in the AS path (1,000,000)¶
• 4. Reject bogon prefixes (997,500)¶
• 5. Reject RPKI invalid NLRI (997,000)¶
• 6. Accept only prefixes in the neighbor's cone (992,000)¶

In this list, rules 1-5 will be executed for most NLRI seen from the neighbor. In total, 5,986,500 operations are executed on the received NLRI, even though ultimately only 60 prefixes should be imported.¶

While most hardware implementations of BGP speakers should be sufficiently equipped with resources to handle such individual spikes, practice shows that operators can not always use BGP speakers with an abundance of resources. Furthermore, even more well equipped platforms may suffer if multiple neighbors coordinate and utilize this mechanic to induce load. Ultimately, it is also desirable to reduce unnecessary computation independent of security considerations.¶

Hence, it is RECOMMENDED that operators structure rulesets in a way that prioritized early decisions on the majority of routes. For the example above, this would mean, again noting the number of operations per rule:¶

• 1. Reject prefixes not in the neighbor's cone (1,000,000)¶
• 2. Reject bogon prefixes (60)¶
• 3. Reject NLRI with private AS numbers in the AS path (60)¶
• 4. Reject RPKI invalid NLRI (60)¶
• 5. Add community imported from peer (60)¶
• 6. Add community imported in LOCATION (60)¶

Overall, this reduces the number of operations for our hypothetical full-table from 5,986,500 operations to 1,000,300. Similar effects can occur when not filtering on, e.g., the OTC attribute first when sending prefixes to customers.¶

8.1.3. Ensuring Consistency when Changing Prefixfilters

As discussed in Section 8.1.2, many BGP implementations use a sequential order for applying different prefix filters to ingested routes. However, at the same time, several implementations do not perform atomic operations when applying rules. This means that, especially on resource constraint BGP speakers or BGP speakers under load consistency of a ruleset may be lost during a rule-set update.¶

For example, consider the following simplified export rule-set towards a peer:¶

• 1. Reject prefixes learned from upstreams¶
• 2. Reject bogon prefixes¶
• 3. Reject RPKI invalid NLRI¶
• 4. Accept all remaining prefixes¶

If one now wants to swap the order of Rule 2 and 3, an implementation applying rule updates not atomically would proceed as follows:¶

• 1. Delete Rule 2, Reject bogon prefixes¶
• 2. Add Rule 2, Reject RPKI invalid NLRI¶
• 3. Delete Rule 3, Reject RPKI invalid NLRI¶
• 4. Add Rule 3, Reject bogon prefixes¶

During the timeframe between the execution of Step 1 and 2, an NLRI for a bogon prefix would be passed by the filter. While, technically, this timeframe should be negligibly small, a loaded control plane my create unexpected overhead allowing prefixes that should be filtered to pass. Similarly, an error during the application of a ruleset, making the application stop after the execution of Step 1 may have a similar effect if rule-set changes are not atomic.¶

Hence, it is RECOMMENDED that operators assess whether the application of changes to rule-sets on their BGP speakers is atomic. If it is not atomic, operators SHOULD take special care in drafting rule-set updates concerning inconsistent state that could be created by a delayed or incomplete update. If no atomicity is provided by the BGP speaker, and the load-conditions are uncertain, operators SHOULD consider creating a new complete rule-set with the desired changes, and then changing the referenced rule-set for a given neighbor instead of updating an existing rule-set in-place. Naturally, after the new rule-set has been activated, the old rule-set should be deleted.¶

8.1.4. Ensuring Idempotency for Prefixfilter Changes

As prefix filters are changed regularly, idempotency is essential when issuing automated updates of prefix filters. Specifically, prefix filters SHOULD NOT be generated on routers itself.¶

Instead, filter lists SHOULD be generated on dedicated systems. These systems SHOULD ensure the idempotency of changes to filters applied to routers, i.e., they should only deploy a policy, if the policy changed. This ensures no unnecessary regular load is placed on the control plane of BGP speakers.¶

8.1.5. Ruleset Size Considerations

Some BGP speaker implementations, and especially older BGP speakers, are restrained in terms of the number of prefixes and rules they can apply. A common reaction of operators in such cases is reducing the number of filters applied on sessions. Even though it is NOT RECOMMENDED to aggregate prefix lists for filtering, operators SHOULD consider aggressive aggregation of prefix filter lists to restrict the perfixes accepted by neighbors if the alternative is not using filters at all.¶

Another approach that MAY be a suitable rule-set creation approach for downstreams and peers is offline validation. In that case, a dedicated system regularly, e.g., every two hours, obtains the list of prefixes advertised by a given peer or downstream. That list is then validated according to the applicable section below. Subsequently, instead of using a full representation of the neighbors cone, a condensed prefix list matching the aggregate of the exact prefixes announced is generated and deployed to the BGP speaker. While this increases the timeframe for newly added prefixes to be accepted, and may be unsuitable for, e.g., DDoS defense services, it can also reduce the size of prefix lists significantly.¶

8.1.6. Ruleset Generation Failure

As noted in Section 6.1.3, the creation and application of filter rules should be automated to reduce the margin for error and misconfigurations. Nevertheless, the regeneration of filter rules may fail.¶

Before applying a generated ruleset, an operator should check it for obvious errors and potentially require manual intervention to remediate the issue. Examples include a ruleset for a neighbor suddenly significantly increasing or decreasing in size, or being empty.¶

In case of such a failure, each administrator MAY decide which actions they will take. Options include re-using the previously active rule set, or either accepting or rejecting all routes depending on routing policy. Generally accepting all routes during that time frame could break BGP routing security. However, rejecting them might re-route too much traffic towards upstreams, and could cause more harm than accepting invalid prefixes. Similarly, reusing the previously active rule set may lead to prefixes being wrongfully accepted or rejected, despite on a smaller scale than for a general accept or reject decision.¶

Hence, to still provide sufficient protection for an individual AS experiencing issues with rule generation, and therefore deciding to deviate to more permissive inbound filters, it is strongly RECOMMENDED that all BGP speakers in general employ inbound and outbound filtering as described in this document.¶

8.2.1. Filters with Internet Peers

8.2.2. Filters with Customers

8.2.3. Filters with Upstream Providers

8.3.1. Inbound Filtering

The leaf network will deploy the filters corresponding to the routes it is requesting from its upstream. If a default route is requested, a simple inbound filter can be applied to accept only the default route (Section 5.4). If the leaf network is not capable of listing the prefixes because there are too many (for example, if it requires the full Internet routing table), then it SHOULD follow the filter recommendations in Section 8.2.3.1.¶

8.3.2. Outbound Filtering

A leaf network will most likely have a very straightforward policy: it SHOULD only announce its local routes. For additional scrutiny, it is also RECOMMENDED that leaf ASes follow the recommendations in Section 8.2.3.2 to avoid announcing invalid routes to its upstream provider, and for additional resillience if the network later becomes multihomed.¶