| Internet-Draft | Framework of OAM for DetNet | January 2024 |
| Mirsky, et al. | Expires 11 July 2024 | [Page] |
Deterministic Networking (DetNet), as defined in RFC 8655, aims to provide bounded end-to-end latency on top of the network infrastructure, comprising both Layer 2 bridged and Layer 3 routed segments. This document's primary purpose is to detail the specific requirements of the Operation, Administration, and Maintenance (OAM) recommended to maintain a deterministic network. The document will be used in future work that defines the applicability of and extension of OAM protocols for a deterministic network. With the implementation of the OAM framework in DetNet, an operator will have a real-time view of the network infrastructure regarding the network's ability to respect the Service Level Objective, such as packet delay, delay variation, and packet loss ratio, assigned to each DetNet flow.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 11 July 2024.¶
Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Deterministic Networking (DetNet) [RFC8655] has proposed to provide a bounded end-to-end latency on top of the network infrastructure, comprising both Layer 2 bridged and Layer 3 routed segments. That work encompasses the data plane, OAM, time synchronization, management, control, and security aspects.¶
Operations, Administration, and Maintenance (OAM) Tools are of primary importance for IP networks [RFC7276]. DetNet OAM should provide a toolset for fault detection, localization, and performance measurement.¶
This document's primary purpose is to detail the specific requirements of the OAM features recommended to maintain a deterministic/reliable network. Specifically, it investigates the requirements for a deterministic network, supporting critical flows.¶
In this document, the term OAM will be used according to its definition specified in [RFC6291]. DetNet expects to implement an OAM framework to maintain a real-time view of the network infrastructure, and its ability to respect the Service Level Objectives (SLOs), such as in-order packet delivery, packet delay, delay variation, and packet loss ratio, assigned to each DetNet flow.¶
This document lists the functional requirements toward OAM for a DetNet domain. The list can further be used for gap analysis of available OAM tools to identify possible enhancements of existing or whether new OAM tools are required to support proactive and on-demand path monitoring and service validation.¶
This document uses definitions, particularly of a DetNet flow, provided in Section 2.1 of [RFC8655]. The following terms are used throughout this document as defined below:¶
OAM: Operations, Administration, and Maintenance¶
DetNet: Deterministic Networking¶
PREOF: Packet Replication, Elimination and Ordering Functions¶
SLO: Service Level Objective¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. The requirements language is used in Section 1.1, Section 6, and applies to the implementations of DetNet OAM.¶
DetNet networks expect to provide communications with predictable low packet delay, packet loss, and packet misordering. Most critical applications will define a set of SLOs to be required for the DetNet flows it generates.¶
To respect strict guarantees, DetNet can use an orchestrator able to monitor and maintain the network. Typically, a Software-Defined Network controller places DetNet flows in the deployed network based on their SLOs. Thus, resources have to be provisioned a priori for the regular operation of the network.¶
Most of the existing OAM tools can be used in DetNet networks, but they can only cover some aspects of deterministic networking. Fulfilling strict guarantees is essential for DetNet flows, resulting in new DetNet-specific functionalities that must be covered with OAM. Filling these gaps is inevitable and needs accurate consideration of DetNet specifics. Similar to DetNet flows, their OAM also needs careful end-to-end engineering.¶
For example, appropriate placing of MEPs along the path of a DetNet flow is not always a trivial task and may require proper design, together with the design of the service component of a given DetNet flow.¶
There are several DetNet-specific challenges for OAM. Bounded network characteristics (e.g., delay, loss) are inseparable service parameters; therefore, Performance Monitoring OAM is a key topic for DetNet. OAM tools are needed to monitor each SLO without impacting the DetNet flow characteristics. A further challenge is strict resource allocation. Resources used by OAM must be considered and allocated to avoid disturbing DetNet flows.¶
The DetNet Working Group has defined two sub-layers:¶
OAM mechanisms exist for the DetNet forwarding sub-layer, but the service sub-layer requires new OAM procedures. These new OAM functions must allow, for example, to recognize/discover DetNet relay nodes, to get information about their configuration, and to check their operation or status.¶
DetNet service sub-layer functions use a sequence number for PREOF. That creates a challenge for inserting OAM packets in the DetNet flow.¶
Fault tolerance also assumes that multiple paths could be provisioned to maintain an end-to-end circuit by adapting to the existing conditions. The DetNet Controller Plane, e.g., central controller/orchestrator, controls the PREOF on a node. OAM is expected to support monitoring and troubleshooting PREOF on a particular node and within the domain.¶
Note that a distributed architecture of the DetNet Control Plane can also control PREOF in those scenarios where DetNet solutions involve more than one single central controller.¶
The DetNet forwarding sub-layer is based on preexisting technologies and has much better coverage regarding OAM. However, the forwarding sub-layer is terminated at DetNet relay nodes, so the end-to-end OAM state of forwarding may be created only based on the status of multiple forwarding sub-layer segments serving a given DetNet flow (e.g., in case of DetNet MPLS, there may be no end-to-end LSP below the DetNet Pseudowire).¶
OAM features will enable DetNet with robust operation both for forwarding and routing purposes.¶
It is worth noting that the test and data packets are expected to follow the same path, i.e., connectivity verification has to be conducted in-band without impacting data traffic. It is expected that test packets share fate with the monitored data traffic without introducing congestion in normal network conditions.¶
Information about the state of the network can be collected using several mechanisms. Some protocols, e.g., Simple Network Management Protocol, polls for updated data. Other protocols, such as YANG-Push [RFC8641], can be used to set up subscriptions for the data defined in the YANG data models to be published periodically or when the underlying data changes. In either way, information is collected and sent using the DetNet Controller Plane.¶
Also, we can characterize methods of transporting OAM information relative to the path of data. For instance, OAM information may be transported in-band or out-of-band relative to the DetNet flow. In case of the former, the telemetry information uses resources allocated for the monitored DetNet flow. If an in-band method of transporting telemetry is used, the amount of generated information needs to be carefully analyzed, and additional resources must be reserved. [RFC9197] defines the in-band transport mechanism where telemetry information is collected in the data packet on which information is generated. Two tracing methods are described - end-to-end, i.e., from the ingress and egress nodes, and hop-by-hop, i.e., like end-to-end with additional information from transit nodes. [RFC9326] and [I-D.mirsky-ippm-hybrid-two-step] are examples of out-of-band telemetry transport. In the former case, information is transported by each node traversed by the data packet of the monitored DetNet flow in a specially constructed packet. In the latter, information is collected in a sequence of follow-up packets that traverse the same path as the data packet of the monitored DetNet flow. In both methods, transport of the telemetry can avoid using resources allocated for the DetNet domain.¶
Continuity check is used to monitor the continuity of a path, i.e., that there exists a way to deliver packets between MEP A and MEP B. The continuity check detects a network failure in one direction, from the MEP transmitting test packets to the remote egress MEP. Continuity check in a DetNet OAM domain monitors the DetNet forwarding sub-layer and thus is not affected by a PREOF that operates at the DetNet service sub-layer ([RFC8655].¶
In addition to the Continuity Check, DetNet solutions have to verify connectivity. This verification considers an additional constraint, i.e, the absence of misconnection. The misconnection error state is entered after several consecutive test packets from other DetNet flows are received. The definition of the conditions for entry and exit of a misconnection error state is outside the scope of this document. Connectivity verification in a DetNet OAM domain monitors the DetNet forwarding sub-layer and thus is not affected by PREOF that operates at the DetNet service sub-layer ([RFC8655].¶
Ping and traceroute are two ubiquitous tools that help localize and characterize a failure in the network using an echo request/reply mechanism. They help to identify a subset of the routers in the path. However, to be predictable, resources are reserved per flow in DetNet. Thus, DetNet needs to define route tracing tools able to trace the route for a specific flow. Also, tracing can be used for the discovery of the Path Maximum Transmission Unit or location of elements of PREOF for the particular route in the DetNet domain.¶
DetNet is not expected to use Equal-Cost Multipath (ECMP) [RFC8939]. As the result, DetNet OAM in an ECMP environment is outside the scope of this document.¶
DetNet expects to operate fault-tolerant networks. Thus, mechanisms able to detect faults before they impact network performance are needed.¶
The network has to detect when a fault has occurred, i.e., the network has deviated from its expected behavior. Fault detection can be based on proactive OAM protocols like continuity check or on-demand methods like ping. While the network must report an alarm, the cause may not be identified precisely. Examples of such alarms are significant degradation of the end-to-end reliability, or a buffer overflow occurs.¶
An ability to localize a network defect and provide its characterization are necessary elements of network operation.¶
Hybrid OAM methods are used in performance monitoring and defined in [RFC7799] as:¶
A hybrid measurement method can produce metrics as close to measured using a passive measurement method. The passive methods measure metrics closest to the network's actual conditions. A hybrid method, even if it alters something in a data packet, even if that is as little as the value of a designated field in the packet encapsulation, is considered an approximation of a passive measurement method. One example of such a hybrid measurement method is the Alternate Marking method (AMM) described in [RFC9341]. As with all on-path telemetry methods, AMM in a DetNet domain with the IP data plane is natively in-band with respect to the monitored DetNet flow. Because the marking is applied to a data flow, measured metrics are directly applicable to the DetNet flow. AMM minimizes the additional load on the DetNet domain by using nodal collection and computation of performance metrics in combination with optionally using out-of-band telemetry collection for further network analysis.¶
The ability to expose a collection of metrics to support an operator's decision-making is essential. The following performance metrics are useful:¶
It is important to optimize the volume and frequency of statistics/measurement collection, whether the mechanisms are distributed, centralized, or both. Periodic and event-triggered collection information characterizing the state of a network is an example of mechanisms to achieve the optimization.¶
DetNet aims to enable real-time communications on top of a heterogeneous multi-hop architecture. To make correct decisions, the DetNet Controller Plane [RFC8655] needs timely information about packet losses/delays for each flow, and each hop of the paths. In other words, just the average end-to-end statistics are not enough. The collected information must be sufficient to allow a system to predict the worst-case scenario.¶
Service protection (provided by the DetNet Service sub-layer) is designed to mitigate simple network failures more rapidly than the expected response time of the DetNet Controller Plane. In the face of events that impact network operation (e.g., link up/down, device crash/reboot, flows starting and ending), the DetNet Controller Plane needs to perform repair and re-optimization actions in order to permanently ensure SLOs of all active flows with minimal waste of resources. The Controller Plane is expected to be able to continuously retrieve the state of the network, to evaluate conditions and trends about the relevance of a reconfiguration, quantifying:¶
Thus, reconfiguration may only be triggered if the gain is significant.¶
When multiple paths are reserved between two MEPs, packet replication may be used to introduce redundancy and alleviate transmission errors and collisions. For instance, in Figure 1, the source device S transmits a packet to devices A and B to reach the destination node R.¶
===> (A) => (C) => (E) ===
// \\// \\// \\
source (S) //\\ //\\ (R) (root)
\\ // \\ // \\ //
===> (B) => (D) => (F) ===
Because the quality of service criteria associated with a path may degrade, the network has to provision additional resources along the path.¶
According to [RFC8655], DetNet functionality is divided into forwarding and service sub-layers. The DetNet forwarding sub-layer includes DetNet transit nodes and may allocate resources for a DetNet flow over paths provided by the underlay network. The DetNet service sub-layer includes DetNet relay nodes and provides a DetNet service (e.g., service protection). This section lists general requirements for DetNet OAM as well as requirements in each of the DetNet sub-layers of a DetNet domain.¶
The OAM functions for the DetNet service sub-layer allow, for example, to recognize/discover DetNet relay nodes, to get information about their configuration, and to check their operation or status.¶
The requirements on OAM for a DetNet relay node are:¶
This document has no actionable requirements for IANA. This section can be removed before publication.¶
This document lists the OAM requirements for a DetNet domain and does not raise any security concerns or issues in addition to ones common to networking and those specific to DetNet discussed in Section 9 of [RFC9055]. Furthermore, the analysis of OAM security concerns in Section 6 of [RFC7276] also applies to DetNet OAM, including the use of OAM for network reconnaissance.¶
Privacy considerations of DetNet discussed in Section 13 of [RFC9055] are also applicable to DetNet OAM. If any privacy mechanism is used for the monitored DetNet flow, then the same privacy method MUST be applied to the active DetNet OAM used to monitor the flow.¶
The authors express their appreciation and gratitude to Pascal Thubert for the review, insightful questions, and helpful comments.¶