| TOC |
|
By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”
The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 16, 2008.
This draft describes operational procedures aimed at reducing the amount of traffic lost during planned maintenances of routers, involving the shutdown of BGP peering sessions.
1.
Introduction
2.
Terminology
3.
Packet loss upon manual eBGP session shutdown
4.
Practices to avoid packet losses
4.1.
Improving availability of alternate paths
4.2.
Graceful shutdown procedures for eBGP sessions
4.2.1.
Outbound traffic
4.2.2.
Inbound traffic
4.3.
Graceful shutdown procedures for iBGP sessions
5.
Forwarding modes and forwarding loops
6.
Dealing with Internet policies
7.
Effect of the g-shut procedure on the convergence
7.1.
Maintenance of an eBGP session
7.1.1.
Propagation on the other eBGP sessions of the g-shut initiator
7.1.2.
Propagation on the other iBGP sessions of the g-shut initiator
7.1.3.
Propagation of updates in an iBGP full-mesh
7.1.4.
Propagation of updates from iBGP to iBGP in a RR hierarchy
7.2.
Maintenance of an iBGP session
7.3.
Applicability of the g-shut procedures
7.4.
Summary of operations
7.4.1.
Pre-configuration
7.4.2.
Operations at maintenance time
8.
Techniques with limited applicability
8.1.
In-filter reconfiguration
8.2.
Multi Exit Discriminator tweaking
8.3.
IGP distance Poisoning
9.
IANA considerations
10.
Security Considerations
11.
References
§
Authors' Addresses
§
Intellectual Property and Copyright Statements
| TOC |
Routing changes in BGP can be caused by planned, manual, maintenance operations. This document discusses operational procedures to be applied in order to reduce or eliminate losses of packets during the maintenance. These losses come from the transient lack of reachability during the BGP convergence following the shutdown of an eBGP peering session between two Autonomous System Border Routers (ASBR).
This document presents procedures for the cases where the forwarding plane is impacted by the maintenance, hence when the use of Graceful Restart does not apply.
The procedures described in this document can be applied to reduce or avoid packet loss for outbound and inbound traffic flows initially forwarded along the peering link to be shut down. These procedures allow routers to keep using old paths until alternate ones are learned, ensuring that routers always have a valid route available during the convergence process.
The goal of the document is to meet the requirements described in [REQS] (Decraene, B., Francois, P., Pelsser, C., and Z. Ahmad, “Requirements for the graceful shutdown of BGP sessions,” December 2007.) at best without changing the BGP protocol or BGP implementations.
Still, it explains why standardizing a community value for the purpose of BGP session graceful shutdown would reduce the management overhead bound with the solution. It would also allow vendors to provide an automatic graceful shutdown mechanism that does not require any configuration at maintenance time.
| TOC |
g-shut initiator : a router on which the session shutdown is performed for the maintenance.
g-shut neighbor : a router that peers with the g-shut initiator via (one of) the session(s) to be shut down.
Initiator AS : the Autonomous System of the g-shut initiator.
Neighbor AS : the Autonomous System of the g-shut neighbor.
Affected path / Nominal / pre-convergence path : a BGP path via the peering link(s) undergoing the maintenance. This path will no longer exist after the shutdown.
Affected prefix : a prefix initially reached via an affected path.
Affected router : a router having an affected prefix.
Backup / alternate / post-convergence path : a path toward an affected prefix that will be selected as the best path by an affected router, when the link is shut down and the BGP convergence is completed.
Transient alternate path : a path towards an affected prefix that may be transiently selected as best by an affected router during the convergence process but that is not a post-convergence path.
Loss of Connectivity (LoC) : the state when a router has no path towards an affected prefix.
| TOC |
Packets can be lost during a manual shutdown of an eBGP session for two reasons.
First, routers involved in the convergence process can transiently lack of paths towards an affected prefix, and drop traffic destined to this prefix. This is because alternate paths can be hidden by nodes of an AS. This happens when the paths are not selected as best by the ASBR that receive them on an eBGP session, or by Route Reflectors that do not propagate them further in the iBGP topology because they do not select them as best.
Second, within the AS, routers' FIB can be transiently inconsistent during the BGP convergence and packets towards affected prefixes can loop and be dropped. Note that these loops only happen when BR-to-BR encapsulation is not used within the AS.
This document only addresses the first reason.
| TOC |
This section describes means for an ISP to reduce the transient loss of packets upon a manual shutdown of a BGP session.
| TOC |
Using advertise-best-external on ASBRs improves the availability of alternate paths. Hence it reduces the LoC duration for the outbound traffic of the ISP upon an eBGP Session shutdown.
All solutions that improve the availability of paths beyond the Route Reflectors barrier also help in reducing the LoC. These have been discussed for years but are not yet available, probably due to their implementation complexity.
| TOC |
This section aims at describing a procedure to be applied to reduce the LoC with readily available BGP features, and without assuming particular iBGP design for the AS performing the maintenance and in the neighboring ASes.
| TOC |
This section discusses a mean to render the affected paths less desirable by the BGP decision process of affected routers, still allowing these to be used during the convergence while alternate paths are propagated to the affected routers.
A decrease of the local-pref value of the affected paths can be issued in order to render the affected paths less preferable, at the highest possible level of the BGP Decision Process.
This operation can be performed by reconfiguring the out-filters associated with the iBGP sessions established by the g-shut initiator.
The modification of the filters MUST supplant any other rule affecting the local-pref value of the old paths.
Compared to using an in-filter of the eBGP session to be shut down, the modification of the out-filters will not let the g-shut initiator switch to another path, as the input to the BGP decision process of that router does not change. As a consequence, the g-shut initiator will not send a withdraw message over its iBGP sessions when it receives an alternate path over an iBGP session. It will however modify the local-pref of the affected paths so that upstream routers will switch to alternate ones.
When the actual shutdown of the session is performed, the g-shut initiator will itself switch to the alternate paths.
| TOC |
The solution described for the outbound traffic can be applied at the neighbor AS. This can be done either "manually" or by using a community value dedicated to this task.
| TOC |
The operator performing the maintenance of the eBGP session can contact the operator at the other side of the peering link, and let him apply the procedure described above for its own outbound traffic.
| TOC |
A community value (referred to as GSHUT community in this document) can be agreed upon by neighboring ASes. A path tagged with this community must be considered as soon to be affected by a maintenance operation.
| TOC |
A g-shut neighbor is pre-configured to set a low local-pref value for the paths received over eBGP sessions which are tagged with the GSHUT community.
This rule must supplant any other rule affecting the local-pref value of the paths.
This local-pref reconfiguration SHOULD be performed at the out-filters of the iBGP sessions of the g-shut neighbor. That is, the g-shut neighbor does not take into account this low local-pref in its own BGP best path selection. As described in Section 4.2.1 (Outbound traffic) this avoids sending the withdraw messages that can lead to LoC.
| TOC |
Upon the manual shutdown, the output filter associated with the maintained eBGP session will be modified on the g-shut initiator so as to tag all the paths advertised over the session with the GSHUT community.
| TOC |
If the GSHUT community is an extended community, it SHOULD be set non transitive.
If a normal community is used, this community SHOULD be removed from the path by the ASBR of the peer receiving it. If not, the GSHUT community MAY be removed from the path by all the ASBRs of the neighboring AS, before propagating the path to other peers.
Not propagating the community further in the Internet reduces the amount of BGP churn and avoids rerouting in distant ASes that would also recognize this community value. In other words, it helps concealing the convergence at the maintenance location.
There are cases where an interdomain exploration is to be performed to recover the reachability, e.g., in the case of a shutdown in confederations where the alternate paths will be found in another AS of the confederation. In such scenarios, the community value SHOULD be allowed to transit through the confederation but MAY be removed from the paths advertised outside of the confederation.
When the local-pref value of a path is conserved upon its propagation from one AS of the confederation to the other, there is no need to have the GSHUT community be propagated throughout that confederation.
| TOC |
From a configuration burden viewpoint, it would be much easier to have the GSHUT community value be standardized.
First, on the g-shut initiator, an operator would have a single configuration rule to be applied at the maintenance time, which would not depend on the identity of its peer. This would make the maintenance operations less error prone.
Second, on the g-shut neighbor, the same filter related to g-shut can be applied to all iBGP sessions, at the g-shut neighbor.
| TOC |
If the iBGP topology is viable after the maintenance of the session, i.e, if all BGP speakers of the AS have an iBGP signaling path for all affected prefixes after the convergence, then the shutdown of an iBGP session does not lead to transient unreachability.
However, in the case of a shutdown of a router, a reconfiguration of the out-filters of the g-shut initiator should be performed to set a low local-pref value for the paths originated by the g-shut initiator (e.g, BGP aggregates redistributed from other protocols, including static routes).
This behavior is equivalent to the recommended behavior for paths "redistributed" from eBGP sessions to iBGP sessions in the case of the shutdown of an ASBR.
| TOC |
If the AS applying the solution does not rely on encapsulation to forward packets from the Ingress Border Router to the Egress Border Router, then transient forwarding loops and consequent packet losses can occur during the convergence process, even if the procedure described above is applied.
Using the out-filter reconfiguration avoids the forwarding loops between the g-shut initiator and its upstream neighbors. Indeed, when this reconfiguration is applied, the g-shut initiator keeps using its own external path and lets the upstream routers converge to the alternate ones. During this phase, no forwarding loops can occur between the g-shut initiator and its upstream neighbors as the g-shut neighbor keeps using the affected paths via its eBGP peering links. When the first step is finished, all the upstream routers have switched to alternate paths and the transition performed by the g-shut initiator will be loopfree. Transient forwarding loops between other routers will not be avoided with this procedure.
| TOC |
A side gain of the maintenance solution is that it can be used to reduce the churn implied by a shutdown of an eBGP session.
For this, it is recommended to apply the filters modifying the local-pref value of the paths to values strictly lower but as close as possible to the local-pref values of the post-convergence paths.
For example, if a peering link is shut down between a provider and one of its customers, and another peering link with this customer remains active, then the value of the local-pref of the old paths SHOULD be decreased to the smallest possible value of the 'customer' local_pref range, minus 1. Thus, routers will not transiently switch to paths received from shared-cost peers or providers, which could lead to the sending of withdraw messages over eBGP sessions with shared-cost peers and providers.
Proceeding like this reduces both BGP churn and traffic shifting as routers will less likely switch to transient paths.
In our example, it also prevents transient unreachabilities in the neighboring AS that are due to the sending of "abrupt" withdraw messages to shared-cost peers and providers.
| TOC |
This section describes the effect of applying the solution.
| TOC |
This section describes the effect of applying the solution for the shutdown of an eBGP session.
| TOC |
Nothing is propagated on the other eBGP sessions when the out-filters reconfiguration step is applied. The reconfiguration is indeed only defined for its iBGP sessions.
The reconfiguration of the iBGP out-filters will trigger the reception of alternate paths at the g-shut initiator. As the eBGP in-filters have not been modified at that step, the old paths are still preferred by the g-shut initiator.
| TOC |
During the out-filter reconfiguration, path updates are propagated with a reduced local-pref value for the affected paths. As a consequence, Route Reflectors and distant ASBRs select and propagate alternate paths through the iBGP topology as they no longer select the old paths as best.
When the shut-down is performed, for each affected prefix, the g-shut initiator propagates on its iBGP sessions:
. The alternate path, if the best path was received over an eBGP sessions.
. A withdraw, if the best path was received over an iBGP sessions.
| TOC |
No transient LoC can occur if a reconfiguration of the iBGP out-filters on the g-shut initiator is performed.
| TOC |
Upon the reception of the update of a primary path with a lower local-pref value, Route Reflectors will either propagate the update, or select an alternate path and propagate it within their RR iBGP full-mesh, to their own Route Reflectors in the case of a multiple level Route Reflector hierarchy, and to their clients.
During the convergence process controlled with the described procedure, some corner case timings can trigger transient unreachabilities.
A typical example for such transient unreachability for a given prefix is the following :
1. A Route Reflector RR1 only knew about the primary path upon the shutdown.
2. A member of its RR full-mesh, RR2, propagates an update of the old path with a lower local-pref.
3. Another member of its RR full-mesh, RR3 processes the update, selects an alternate path, and propagates an update in the mesh.
4. RR2 receives the alternate path, selects it as best, and hence withdraws the updated old path on the iBGP session of the mesh.
5. If for any reason, RR1 receives and processes the withdraw generated in step 4 before processing the update generated in step 3, RR1 transiently suffers from unreachability for the affected prefix.
In such corner cases, the solution improves the iBGP convergence behavior/LoC but does not ensure 0 packet loss, as we cannot define a simple solution relying only on a reconfiguration of the filters of the g-shut initiator. Improving the availability of alternate paths in Route Reflectors, using advertise best external, or any flavor of add-path [AddPath] (D. Walton, A. Retana, and E. Chen, “Advertisement of Multiple Paths in BGP,” .), seems to be the most pragmatic solution to these corner cases.
| TOC |
If the shutdown does not temper with the viability of the iBGP topology, the described procedure is sufficient to avoid LoC.
| TOC |
The applicability of the procedure described in this draft to the cases presented in [REQS] (Decraene, B., Francois, P., Pelsser, C., and Z. Ahmad, “Requirements for the graceful shutdown of BGP sessions,” December 2007.) can be shown by combining the effects described in this section. A complete case by case analysis will be provided in the next versions of the draft.
| TOC |
This section summarizes the configurations and actions to be performed to support the g-shut procedure for eBGP peering links.
| TOC |
On each ASBR supporting the g-shut procedure, set-up an out-filter applied on all iBGP sessions of the ASBR, that :
. sets the local-pref of the paths tagged with the g-shut community to a low value
. removes the g-shut community from the path.
| TOC |
On the g-shut initiator :
. Apply an in-filter on the maintained BGP session to tag the paths received over the session with the g-shut community.
. Apply an out-filter on the maintained BGP session to tag the paths propagated over the session with the g-shut community.
. Wait for convergence to happen.
. Perform a BGP session shutdown.
| TOC |
| TOC |
An In-filter reconfiguration on the eBGP session undergoing the maintenance could be performed instead of out-filter reconfigurations on the iBGP sessions of the g-shut initiator.
Upon the application of the maintenance procedure, if the g-shut initiator has an alternate path in its Adj-Rib-In, it will switch to it directly.
If this new path was advertised by an eBGP neighbor of the g-shut initiator, the g-shut initiator will send a BGP Path Update message over its iBGP and eBGP sessions.
If this new path was received over an iBGP session, the g-shut initiator will select that path and directly propagate a withdraw message over the iBGP sessions for which it is not acting as a Route Reflector. There can be iBGP topologies where the iBGP peers of the g-shut initiator do not know an alternate path, and hence may drop traffic.
Also, applying an In-filter reconfiguration on the eBGP session undergoing the maintenance may lead to transient LoC, in full-mesh iBGP topologies if
a. An ASBR of the initiator AS, ASBR1 did not initially select its own external path as best, and
b. An ASBR of the initiator AS, ASBR2 propagates an Update message along its iBGP sessions upon the reception of ASBR1's update following the in-filter reconfiguration on the g-shut initiator, and
c. ASBR1 receives the update message, runs its Decision Process and hence propagates a withdraw of its external path after having selected ASBR2's path as best, and
d. An impacted router of the AS processes the withdraw of ASBR1 before processing the update from ASBR2.
Applying a reconfiguration of the out-filters prevents such transient unreachabilities.
Indeed, when the g-shut initiator propagates an update of the old path first, the sending of the withdraw by ASBR2 does not trigger unreachability in other nodes as the old path is still available. Indeed, even though it receives alternate paths, the g-shut initiator keeps using its old path as best as the in-filter of the maintained eBGP session has not been modified yet.
Applying the out-filter reconfiguration also prevents packet loops between the g-shut initiator and its direct neighbors when encapsulation is not used between the ASBRs of the AS.
| TOC |
The MED attribute of the paths to be avoided can be increased so as to force the routers in the neighboring AS to select other paths.
The solution only works if the alternate paths are as good as the initial ones with respect to the Local-Pref value and the AS Path Length value. In the other cases, increasing the MED value will not have an impact on the decision process of the routers in the neighboring AS.
| TOC |
The distance to the BGP nexthop corresponding to the maintained session can be increased in the IGP so that the old paths will be less preferred during the application of the IGP distance tie-break rule. However, this solution only works for the paths whose alternates are as good as the old paths with respect to their Local-Pref value, their AS Path length, and their MED value.
Also, this poisoning cannot be applied when nexthop self is used as there is no nexthop specific to the maintained session to poison in the IGP.
| TOC |
Applying the g-shut procedure is rendered much easier when the g-shut community value is to be standardized. Hence this draft suggests to reserve a community value, e.g., 0xFFFF0005, for this purpose.
| TOC |
By providing the g-shut service to a neighboring AS, an ISP provides means to this neighbor to lower the local-pref value assigned to the paths received from this neighbor.
The neighbor could abuse the technique and do inbound traffic engineering by declaring some prefixes as undergoing a maintenance so as to switch traffic to another peering link.
If this behavior is not tolerated by the ISP, it SHOULD monitor the use of the g-shut community by this neighbor.
| TOC |
| [AddPath] | D. Walton, A. Retana, and E. Chen, “Advertisement of Multiple Paths in BGP,” draft-walton-bgp-add-paths-05.txt (work in progress). |
| [REQS] | Decraene, B., Francois, P., Pelsser, C., and Z. Ahmad, “Requirements for the graceful shutdown of BGP sessions,” draft-decraene-bgp-graceful-shutdown-requirements-00.txt , December 2007. |
| TOC |
| Pierre Francois | |
| Universite catholique de Louvain | |
| Place Ste Barbe, 2 | |
| Louvain-la-Neuve 1348 | |
| BE | |
| Email: | pierre.francois@uclouvain.be |
| URI: | http://inl.info.ucl.ac.be/pfr |
| Bruno Decraene | |
| France Telecom | |
| 38-40 rue du General Leclerc | |
| 92794 Issi Moulineaux cedex 9 | |
| FR | |
| Email: | bruno.decraene@orange-ftgroup.com |
| Cristel Pelsser | |
| NTT Corporation | |
| 9-11, Midori-Cho 3 Chrome | |
| Musashino-Shi, Tokyo 180-8585 | |
| JP | |
| Email: | pelsser.cristel@lab.ntt.co.jp |
| TOC |
Copyright © The IETF Trust (2008).
This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an “AS IS” basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.