| Internet-Draft | Network Working Group | July 2021 |
| Li, et al. | Expires 27 January 2022 | [Page] |
The end-to-end network one-way delay is an important performance metric in the 5G network. For realizing the accurate one-way delay measurement, existing methods requires the end-to-end deployment of accurate clock synchronization mechanism, such as PTP or GPS, which results in relatively high deployment cost. Another method can derive the one-way delay from the round-trip delay. In this case, since the delay of the downlink and uplink of the 5G network may be asymmetric, the measurement accuracy is relatively low. Hence, this document introduces a method to measure the end-to-end network one-way delay based on a reference delay guaranteed by deterministic networking without clock synchronization.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 27 January 2022.¶
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶
With the gradual promotion of new-generation network technologies (such as 5G networks) and their application in various industries, SLA guarantees for network quality become more and more important. For example, different 5G services have different requirements for network performance indicators such as delay, jitter, packet loss, and bandwidth. Among them, the 5G network delay is defined as end-to-end one-way delay of the network. Real-time and accurate measurement of the end-to-end one-way delay is very important for the SLA guarantee of network services, and has become an urgent and important requirement.¶
As shown in figure 1, 5G network HD video surveillance service is a common scenario having requirement of end-to-end one-way delay measurement. In this case, one end of the network is a high-definition surveillance camera in the wireless access side, and the other end of the network is a video server. The end-to-end one-way delay from the surveillance camera to the video server is the sum of T1, T2, T3 and T4, which is composed of delay in wireless access network, optical transmission network, 5G core network, and IP data network.¶
+--------+ +-------+ +-------+ +-------+
+------+ |Wireless| |Optical| |5G Core| | IP | +------+
|Camera+<->+ Access +<->+ Trans +<->+Network+<->+ Data +<->+Server|
+------+ |Network | |Network| | | |Network| +------+
+--------+ +-------+ +-------+ +-------+
|<---- T1 ---->|<--- T2 -->|<--- T3 -->|<--- T4 ---->|
The existing one-way delay measurement solutions are divided into two types. One type of mechanism to calculate one-way delay is based on the measurement of round-trip delay. However, for example, because upstream traffic and downstream traffic do not share the same path in 5G network, the accuracy of the end-to-end one-way delay calculated from the round-trip delay is low. Another type of mechanism is in-band OAM with accurate network time synchronization mechanism , such as NTP[RFC5905] or PTP[IEEE.1588.2008].¶
The one-way delay measurement solution based on precise network time synchronization requires the deployment of an end-to-end time synchronization mechanism. The current time synchronization accuracy based on the NTP protocol can only reach millisecond level, which cannot fully meet the measurement accuracy requirements. The time synchronization accuracy based on the GPS module or the PTP protocol can meet the requirements. However, because many data centers are actually located underground or in rooms without GPS signals, so GPS clock information cannot be continuously obtained for time synchronization. For time synchronization solutions based on the PTP protocol, each device in the wireless access network, 5G transport network, and 5G core network must support the PTP protocol, which is unrealistic at the moment. So the one-way delay measurement solution based on precise end-to-end time synchronization is expensive and difficult to be deployed.¶
This document introduces a one-way delay measurement mechanism for Deterministic Networking (DetNet) [RFC8655]. The one-way delay measurement is based on a stable one-way delay of a reference DetNet packet, named as reference delay, which is known in advance and has extremely low jitter. We can use the reference delay provided by the reference DetNet packet to derive the one-way delay of other common service packets.¶
NTP Network Time Protocol¶
PTP Precision Time Protocol¶
SLA Service Level Agreement¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14[RFC2119][RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The end-to-end one-way delay of a reference packet with a stable delay in the network can be used as a reference delay, denoted as Dref, which is known in advance and has extremely low jitter. This section will describe in detail the end-to-end one-way delay measurement method based on reference delay of the reference packet. Assume that the end-to-end one-way delay from the sender to the receiver is measured, as shown in figure 2. The intermediate network devices other than the sender and receiver are hidden in the figure.¶
+------------+ +------------+
| | Clock Offset | |
| Sender | +---------------> | Receiver |
| | | |
+------------+ +------------+
Reference +-----+ Dref +-----+
Packet: | Ts1 | +-------------------> | Tr1 |
+-----+ +-----+
Target +-----+ Dtarget +-----+
Packet: | Ts2 | +-------------------> | Tr2 |
+-----+ +-----+
The measurement steps are shown in figure 3, which describe the measurement steps at the sender side and receiver side respectively. For the sender side, a reference packet is sent. In the first step, the sender gets ready to send a reference packet; in the second step, the sender marks an egress timestamp Ts1 for the reference packet; in the third step, the sender encapsulates the egress timestamp of the reference packet in the measurement header of the reference packet; in the fourth step, the sender sends the reference packet. For the target packet, the sender side procedures are the same,we omit it for simplicity. The sending time of the target packet is according to the traffic model of real applications. On the other hand, the sender can send the reference packet according to a fixed frequency or adjust the sending frequency according to the link usage rate, so that the target packet can always find a nearby reference packet to make sure that the sending time interval between the reference packet and the target packet is small.¶
For the reference packet, the processing steps at the receiver are shown in figure 3. In the first step, the reference packet arrives at the receiver, and the receiver receives the reference packet; in the second step, the receiver timestamps the reference packet at the entrance, which is denoted as Tr1; in the third step, the receiver decapsulates the measurement header of the reference packet to obtain the sender side timestamp Ts1; in the fourth step, the receiver records the timestamp information of Ts1 and Tr1; in the fifth step, the receiver uses the source/destination pair obtained by decapsulation in the third step as the search key, queries the reference delay table and records the reference delay search result, denoted as Dref.¶
For the target packet, the processing steps at the receiver are also shown in figure 3. In the first step, the target packet arrives at the receiver, and the receiver receives the target packet; in the second step, the receiver timestamps the target packet at the entrance, which is denoted as Tr2; in the third step, the receiver decapsulates the measurement header of the target packet to obtain the sender side timestamp Ts2; in the fourth step, the receiver records the timestamp information of Ts2 and Tr2; in the fifth step, the receiver calculates the target one-way delay, which we want to measure, according to the recorded timestamp information Ts1, Ts2, Tr1, Tr2 and reference delay information Dref. The target one-way delay of the target packet is recorded as Dtarget.¶
Sender Side Procedures for both Reference and Target Packet: +-------+ +------------+ +-------------+ +-------+ |Sender | |Sender Side | |Sender Side | |Sending| |Ready +-->+Timestamping+-->+Encapsulation+-->+ Packet| | | | | | | | | +-------+ +------------+ +-------------+ +-------+ Receiver Side Procedures for Reference Packet: +---------+ +-------------+ +-------------+ +---------+ +---------+ |Reference| |Receiver Side| |Receiver Side| |Timestamp| |Query for| |Packet +->+Timestamping +->+Decapsulation+->+Recorded +->+Reference| |Arrival | | | | | | | | Delay | +---------+ +-------------+ +-------------+ +---------+ +---------+ Receiver Side Procedures for Target Packet: +-------+ +-------------+ +-------------+ +---------+ +-----------+ | Target| |Receiver Side| |Receiver Side| |Timestamp| | One-way | | Packet+->+Timestamping +->+Decapsulation+->+Recorded +->+ Delay | |Arrival| | | | | | | |Calculation| +-------+ +-------------+ +-------------+ +---------+ +-----------+
Now we describe the fifth step of the receiver procedures for the target packet in figure 3, that is, calculating the one-way delay Dtarget of the target packet based on the recorded timestamp information Ts1, Ts2, Tr1, Tr2 and the reference delay information Dref. The calculation method is the core of this solution. For the reference packet, leveraging the receiver timestamp minus the sender timestamp, we can get Equation 1.¶
Equation 1: Tr1 - Ts1 = Dref + Offset1¶
where Offset1 is the time offset between the sender and the receiver when the reference packet transmission occurs. Similarly, for the target packet, we can get Equation 2 using the same method.¶
Equation 2: Tr2 - Ts2 = Dtarget + Offset2¶
where Offset2 is the time offset between the sender and the receiver when the target packet transmission occurs. Assuming that the sending time interval between the reference packet and the target packet is very small, we can get that Offset1 = Offset2. By (Equation 2 - Equation 1), we can get Equation 3.¶
Equation 3: Dtarget = (Tr2 + Ts1) - (Tr1 + Ts2) + Dref¶
So the one-way delay of the target packet can be calculated by Equation 3.¶
The sender encapsulates the timestamp information and sender-receiver pair information in the measurement header of the sent packet, as shown in figure 4. The position of measurement header is in the option field of the TCP protocol header. The delay measurement option format is defined in figure 5. The Length value is 8 octets, which is in accordance with TCP option. The sender ID is one octet, and the receiver ID is also one octet. The sender side timestamp is 4 octets, which can store accurate timestamp information.¶
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | Ethernet header (14 octets) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | IP header (20 octets) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | TCP header (20 octets) | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | TCP Delay Measurement Option (8 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Kind=TBA | Length | Sender ID | Receiver ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sender Side Timestamp (4 octets) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The end-to-end one-way delay includes three parts, namely the transmission delay, the internal processing delay of the network devices, and the internal queueing delay of the network devices. Among them, fixed parts of the delay include transmission delay and internal processing delay. The transmission delay is related to transmission distance and transmission media. For example, in optical fiber, it is about 5ns per meter. With transmission path and media determined, it is basically a fixed value. The internal processing delay of a network device includes processing delay of the device's internal pipeline or processor and serial-to-parallel conversion delay of the interface, which is related to in/out port rate of the device, message length and forwarding behavior. The magnitude of the internal processing delay is at microsecond level, and it is basically a fixed value related to the chip design specifications of a particular network device. Variable part of the delay is the internal queueing delay. The queueing delay of the device internal buffer is related to the queue depth, queue scheduling algorithm, message priority and message length. For each device along the end-to-end path, the queueing delay can reach microsecond or even millisecond level, depending on values of the above parameters and network congestion state.¶
With the continuous development of networking technologies and application requirements, a series of new network technologies have emerged which can guarantee bounded end-to-end delay and ultra small jitter. For example, deterministic network[RFC8655], by leveraging novel scheduling algorithms and packet priority settings, can stabilize queuing delay of network device on the end-to-end path. As a result, the end-to-end one-way delay is extremely low and bounded. So packets transmitted by a deterministic network with delay guarantee can be used as reference packets, and their end-to-end one-way delay can be used as reference delays. The acquisition method of reference delay is not limited to the above method based on deterministic network technology.¶
This document requests IANA to assign a Kind Number in TCP Option to indicate TCP Delay Measurement option.¶