| Internet-Draft | Coding and congestion | February 2022 |
| Kuhn, et al. | Expires 26 August 2022 | [Page] |
Forward Erasure Correction (FEC) is a reliability mechanism that is distinct and separate from the retransmission logic in reliable transfer protocols such as TCP. FEC coding can help deal with losses at the end of transfers or with networks having non-congestion losses. However, FEC coding mechanisms should not hide congestion signals. This memo offers a discussion of how FEC coding and congestion control can coexist. Another objective is to encourage the research community to also consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems.¶
This document is the product of the Coding for Efficient Network Communications Research Group (NWCRG). The scope of the document is end-to-end communications: FEC coding for tunnels is out-of-the scope of the document.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 26 August 2022.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
There are cases where deploying FEC coding improves the performance of a transmission. As an example, it may take time for a sender to detect transfer tail losses (losses that occur at the end of a transfer, where, e.g., TCP obtains no more ACKs that would enable it to quickly repair the loss via retransmission). Allowing the receiver to recover such losses instead of having to rely on a retransmission could improve the experience of applications using short flows. Another example is a network where non-congestion losses are persistent and prevent a sender from exploiting the link capacity.¶
Coding and the loss detection of congestion controls are two distinct and separate reliability mechanisms that is distinct and separate from the loss detection of congestion controls. Since FEC coding repairs losses, blindly applying FEC may easily lead to an implementation that also hides a congestion signal from the sender. It is important to ensure that such information hiding does not occur, because loss may be the only congestion signal available to the sender (e.g. TCP [RFC5681]).¶
FEC coding and congestion control can be seen as two separate channels. In practice, implementations may mix the signals that are exchanged on these channels. This memo offers a discussion of how FEC coding and congestion control coexist. Another objective is to encourage the research community also to consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems. This document does not aim at proposing guidelines for characterizing FEC coding solutions.¶
We consider three architectures for end-to-end unicast data transfer:¶
A typical scenario for the considerations in this document is a client browsing the web or watching a live video.¶
This document represents the collaborative work and consensus of the Coding for Efficient Network Communications Research Group (NWCRG); it is not an IETF product and is not a standard. The document follows the terminology proposed in the taxonomy document [RFC8406].¶
Traffic from or to different end users may share various types of bottlenecks. When such a shared bottleneck does not implement some form of flow protection, the share of the available capacity between single flows can help assess when one flow starves the other.¶
As one example, for residential accesses, the data rate can be guaranteed for the customer premises equipment, but not necessarily for the end user. The quality of service that guarantees fairness between the different clients can be seen as a policy concern [I-D.briscoe-tsvarea-fair].¶
While past efforts have focused on achieving fairness, quantifying and limiting harm caused by new algorithms (or algorithms with coding) is more practical [BEYONDJAIN]. This document considers fairness as the impact of the addition of coded flows on non-coded flows when they share the same bottleneck. It is assumed that the non-coded flows respond to congestion signals from the network. This document does not contribute to the definition of fairness at a wider scale.¶
Figure 1 and Figure 2 present the notations that will be used in this document and introduces the Forward Erasure Correction (FEC) and Congestion Control (CC) channels. The Forward Erasure Correction channel carries repair symbols (from the sender to the receiver) and information from the receiver to the sender (e.g. signaling which symbols have been recovered, loss rate prior and/or after decoding, etc.). The Congestion Control channel carries network packets from a sender to a receiver, and packets signaling information about the network (number of packets received vs. lost, Explicit Congestion Notification (ECN) [RFC3168] marks, etc.) from the receiver to the sender. The network packets that are sent by the Congestion Control channel may be composed of source packets and/or repair symbols.¶
SENDER RECEIVER +------+ +------+ | | ----- network packets ---->| | | CC | | CC | | | <--- network information ---| | +------+ +------+
SENDER RECEIVER +------+ +------+ | | source and/or | | | | ----- repair symbols ---->| | | FEC | | FEC | | | signaling | | | | <--- recovered symbols ----| | +------+ +------+
Inside a host, the CC and FEC entities can be regarded as conceptually separate:¶
| ^ | ^ | source | coding |packets | sending | packets | rate |requirements | rate (or v | v | window) +---------------+source +-----------------+ | FEC |and/or | CC | | |repair | |network | |symbols | |packets +---------------+==> +-----------------+==> ^ ^ | signaling | network | recovered symbols | information
| | | source and/or | network | repair symbols | packets v v +---------------+ +-----------------+ | FEC |signaling | CC | | |recovered | |network | |symbols | |information +---------------+==> +-----------------+==>
Figure 3 and Figure 4 provide more details than Figure 1 and Figure 2. Some elements are introduced:¶
The inputs to FEC (incoming data packets without repair symbols, and signaling from the receiver about losses and/or recovered symbols) are distinct from the inputs to CC. The latter calculates a sending rate or window from network information, and it takes the packet to send as input, sometimes along with application requirements such as upper/lower rate bounds, periods of quiescence, or a priority. It is not clear that the ACK signals feeding into a congestion control algorithm are useful to FEC in their raw form, and vice versa - information about recovered blocks may be quite irrelevant to a CC algorithm.¶
The choice of the adequate transport layer may be related to application requirements and the services offered by a transport protocol [RFC8095]:¶
The transport layer may provide an unreliable transport service (e.g. UDP or DCCP [RFC4340]) or a partially reliable transport service (e.g. SCTP with the partial reliability extension [RFC3758] or QUIC with the unreliable datagram extension [I-D.ietf-quic-datagram]). Depending on the amount of redundancy and network conditions, there could be cases where it becomes impossible to carry traffic. This is further discussed in Section 3 where "FEC above CC" case is assessed and in Section 4 and in Section 5 where "FEC in CC" and "FEC below CC" are assessed.¶
The application layer can be composed of several streams above FEC and transport layers instances. The transport layer can exploit a multipath mechanism. The different streams could exploit different paths between the sender and the receiver. Moreover, a single-stream application could also exploit a multipath transport mechanism. This section describes what is in the scope of this document in regards with multi-streams applications and multipath transport protocols.¶
The different combinations between multi-stream applications and multipath transport are the following: (1) one application layer stream as input packets above a combination of FEC and multipath (Mpath) transport layers (Figure 5), and (2) multiple application layer streams as input packets above a combination of FEC and multipath (Mpath) or single path (Spath) transport layers (Figure 6). This document further details cases I (in Section 3.7), II (in Section 4.6) and III (in Section 5.7) illustrated in Figure 5. Cases IV, V and VI of Figure 6 are related to how multiple streams are managed by a single transport or FEC layer: this does not directly concerns the interaction between FEC and the transport and is out of the scope of this document.¶
CASE I CASE II CASE III
+---------------+ +---------------+ +---------------+
| Stream 1 | | Stream 2 | | Stream 3 |
+---------------+ +---------------+ +---------------+
+---------------+ +---------------+ +---------------+
| FEC | | FEC | |Mpath Transport|
+---------------+ | in | +---------------+
|Mpath Transport|
+---------------+ | | +-----+ +-----+
|Mpath Transport| | | |Flow1|...|FlowM|
+---------------+ +---------------+ +-----+ +-----+
+-----+ +-----+ +-----+ +-----+ +-----+ +-----+
|Flow1|...|FlowM| |Flow1|...|FlowM| | FEC |...| FEC |
+-----+ +-----+ +-----+ +-----+ +-----+ +-----+
CASE IV CASE V CASE VI
+-------+ +-------+ +-------+ +-------+ +-------+ +-------+
|Stream1|...|StreamM| |Stream1|...|StreamM| |Stream1|...|StreamM|
+-------+ +-------+ +-------+ +-------+ +-------+ +-------+
+-------------------+ +-------------------+ +-------------------+
| | | FEC | | Mpath Transport |
| FEC | +-------------------+ +-------------------+
| above/in/below |
| Spath Transport | +-------------------+ +-------------------+
| | | Mpath Transport | | FEC |
+-------------------+ +-------------------+ +-------------------+
+-------------------+ +-----+ +-----+ +-----+ +-----+
| Flow | |Flow1| ... |FlowM| |Flow1| ... |FlowM|
+-------------------+ +-----+ +-----+ +-----+ +-----+
[RFC8406] summarizes recommended terminology for Network Coding concepts and constructs. In particular, the document identifies the following coding types (among many others):¶
The decoding scheme may not be able to decode all the symbols. The chance of decoding the erased packets depends on the size of the encoding window, the coding rate and the distribution of erasure in the transmission channel. The FEC channel may let the client transmit information related to the need of supplementary symbols to adapt the level of reliability. Partial and full reliability could be envisioned.¶
| source ^ source
| packets | packets
v |
+-------------+ +-------------+
|FEC | signaling|FEC |
| | recovered| |
| | symbols| |
| | <==| |
+-------------+ +-------------+
| source ^ ^ source
| and/or | sending | and/or
| repair | rate | repair
| symbols | (or window) | symbols
v | |
+-------------+ +-------------+
|Transport | network|Transport |
|(incl. CC) | information| |
| |network <==| |
| |packets | |
+-------------+==> +-------------+
SENDER RECEIVER
Figure 7 presents an architecture where FEC operates on top of the transport.¶
The advantage of this approach is that the FEC overhead does not contribute to congestion in the network when congestion control is implemented at the transport layer, because the repair symbols are sent following the congestion window or rate determined by the CC mechanism. This can result in improved quality of experience for latency sensitive applications such as Voice over IP (VoIP) or any not-fully reliable services.¶
This approach requires that the transport protocol does not implement a fully reliable in-order data transfer service (e.g., like TCP). QUIC with unreliable datagram extension [I-D.ietf-quic-datagram] is an example of a protocol for which this is relevant. In cases where the partially reliable transport is blocked and a fall-back to a reliable transport is proposed, there is a risk for bad interactions between reliability at the transport level and coding schemes. For reliable transfers, coding usage does not guarantee better performance; instead, it would mainly reduce goodput.¶
The addition of coding within the flow does not influence the interaction between coded and non-coded flows. This interaction would mainly depend on the congestion controls associated with each flow.¶
The congestion control mechanism receives network packets and may not be able to differentiate repair symbols from actual source ones. This differentiation requires a transport protocol providing more than the services described in [RFC8095], in particular specifically indicating what information has been repaired. The relevance of adding coding at the application layer is related to the needs of the application. For real-time applications using an unreliable or partially reliable transport, this approach may reduce the number of losses perceived by the application.¶
The coding rate applied at the application layer mainly depends on the available rate or congestion window given by the congestion control underneath. The coding rate could be adapted to avoid adding overhead when the minimum required data rate of the application is not provided by the congestion control underneath. When the congestion control allows sending faster than the application needs, adding coding can reduce packet losses and improve the quality of experience (provided that an unreliable or partially reliable transport is used).¶
The only case where adding useless repair symbols does not obviously result in reduced goodput is when the application rate is limited (e.g., VoIP traffic). In this case, useless repair symbols would only impact the amount of data generated in the network. Extra data in the network can, however, increase the likelihood of increasing delay and/or packet loss, which could provoke a congestion control reaction that would degrade goodput.¶
Irrespective of the transport protocol, a FEC mechanism does not require to implement a reordering mechanism if the application does not need it. However, if the application needs in-order delivery of packets, a reordering mechanism at the receiver is required.¶
The application may require partial reliability. In this case, the coding rate of a FEC mechanism could be adapted based on inputs from the application and the trade-off between latency and packet loss. Partial reliability impacts the type of FEC and type of codec that can be used, such as discussed in Section 2.5.¶
Whether the transport protocol exploits multiple paths or not does not have an impact on the FEC mechanism.¶
| source ^ source
| packets | packets
v |
+------------+ +------------+
| Transport | | Transport |
| | | |
| +---+ +--+ | signaling| +---+ +--+ |
| |FEC| |CC| | recovered| |FEC| |CC| |
| +---+ +--+ | symbols| +---+ +--+ |
| | <==| |
| |network network| |
| |packets information| |
+------------+ ==> <==+------------+
SENDER RECEIVER
Figure 8 presents an architecture where FEC operates within the transport. The repair symbols are sent within what the congestion window or calculated rate allows, such as in [CTCP].¶
The advantage of this approach is that it allows a joint optimization of CC and FEC. Moreover, the transmission of repair symbols does not add congestion in potentially congested networks but helps repair lost packets (such as tail losses). This joint optimization is the key to prevent flows to consume the whole available capacity. The amount of repair traffic injected should not lead to congestion. As denoted in [I-D.singh-rmcat-adaptive-fec], an increase of the repair ratio should be done conjointly with a decrease of the source sending rate.¶
The drawback of this approach is that it may require specific signaling and transport services that may not be described in [RFC8095]. Therefore, development and maintenance may require specific efforts at both transport and coding level and the design of the solution may end up being complex to suit different deployment needs.¶
For reliable transfers, including redundancy reduces goodput for long transfers but the amount of repair symbols can be adapted, e.g. depending on the congestion window size. There is a trade-off between 1) the capacity that could have been exploited by application data instead of transmitting source packets, and 2) the benefits derived from transmitting repair symbols (e.g. unlocking the receive buffer if it is limiting). The coding ratio needs to be carefully designed. For small files, sending repair symbols when there is no more data to transmit could help to reduce the transfer time. Sending repair symbols can avoid the silence period between the transmission of the last packet in the send buffer and 1) firing a retransmission of lost packets, or 2) the transmission of new packets.¶
Examples of the solution could be to add a given percentage of the congestion window or rate as supplementary symbols, or to send a fixed amount of repair symbols at a fixed rate. The redundancy flow can be decorrelated from the congestion control that manages source packets: a separate congestion control entity could be introduced to manage the amount of recovered symbols to transmit on the FEC channel. The separate congestion control instances could be made to work together while adhering to priorities, as in coupled congestion control for RTP media [RFC8699] in case all traffic can be assumed to take the same path, or otherwise with a multipath congestion window coupling mechanism as in Multipath TCP [RFC6356]. Another possibility would be to exploit a lower than best-effort congestion control [RFC6297] for repair symbols.¶
Specific interaction between congestion controls and coding schemes can be proposed (see Section 4.2 and Section 4.3). If no specific interaction is introduced, the coding scheme may hide congestion losses from the congestion controller and the description of Section 5 may apply.¶
The receiver can differentiate between source packets and repair symbols. The receiver may indicate both the number of source packets received and repair symbols that were actually useful in the recovery process of packets. The congestion control at the sender can then exploit this information to tune congestion control behavior.¶
There is an important flexibility in the trade-off, inherent to the use of coding, between (1) reducing goodput when useless repair symbols are transmitted and (2) helping to recover from losses earlier than with retransmissions. The receiver may indicate to the sender the number of packets that have been received or recovered. The sender may use this information to tune the coding ratio. For example, coupling an increased transmission rate with an increasing or decreasing coding rate could be envisioned. A server may use a decreasing coding rate as a probe of the channel capacity and adapt the congestion control transmission rate.¶
The sender may exploit the information given by the receiver to reduce the number of useless repair symbols, and improve goodput.¶
The application may require in-order delivery of packets. In this case, both FEC and transport layer mechanisms should guarantee that packets are delivered in order. If partial ordering is requested by the application, both the FEC and transport could relax the constraints related to in-order delivery: partial ordering impacts both the congestion control and the type of FEC and type of codec that can be used, mostly at the receiver that may need to implement partial reordering.¶
The application may require partial reliability. The reliability offered by FEC may be sufficient, with no retransmission required. This depends on application needs and the trade-off between latency and loss. Partial reliability impacts the type of FEC and type of codec that can be used, such as discussed in Section 2.5.¶
The sender may adapt the coding rate of each of the single subpaths, whether the congestion control is coupled or not. There is an important flexibility on how the coding rate is tuned depending on the characteristics of each subpath.¶
| source ^ source
| packets | packets
v |
+--------------+ +--------------+
|Transport | network|Transport |
|(including CC)| information| |
| | <==| |
+--------------+ +--------------+
| network packets ^ network packets
v |
+--------------+ +--------------+
| FEC |source | FEC |
| |and/or signaling| |
| |repair recovered| |
| |symbols symbols| |
| |==> <==| |
+--------------+ +--------------+
SENDER RECEIVER
Figure 9 presents an architecture where FEC is applied end-to-end below the transport layer, but above the link layer. Note that it is common to apply FEC at the link layer on one or more of the links that make up the end-to-end path. The application of FEC at the link layer contributes to the total capacity that a link exposes to upper layers, but may not be visible to either the end-to-end sender or receiver, if the end-to-end sender and receiver are separated by more than one link, and therefore is out of scope for this document. This includes the use of FEC on top of a link layer in scenarios where the link is known by configuration. In the scenario considered here, the repair symbols are not visible to the end-to-end congestion controller and may be sent on top of what is allowed by the congestion control.¶
Including redundancy adds traffic without reducing goodput but incurs potential fairness issues. The effective bit-rate is higher than the CC's computed fair share due to the transmission of repair symbols, and losses are hidden from the transport. This may cause a problem for loss-based congestion detection, but it is not a problem for delay-based congestion detection.¶
The advantage of this approach is that it can result in performance gains when there are persistent transmission losses along the path.¶
The drawback of this approach is that it can induce congestion in already congested networks. The coding ratio needs to be carefully designed.¶
Examples of the solution could be to add a given percentage of the congestion window or rate as supplementary symbols, or to send a fixed amount of repair symbols at a fixed rate. The redundancy flow can be decorrelated from the congestion control that manages source packets: a separate congestion control entity could be introduced to manage the amount of recovered symbols to transmit on the FEC channel. The separate congestion control instances could be made to work together while adhering to priorities, as in coupled congestion control for RTP media [RFC8699] in case all traffic can be assumed to take the same path, or otherwise with a multipath congestion window coupling mechanism as in Multipath TCP [RFC6356]. Another possibility would be to exploit a lower than best-effort congestion control [RFC6297] for repair symbols.¶
The coding scheme may hide congestion losses from the congestion controller. There are cases where this can drastically reduce the goodput of non-coded flows. Depending on the congestion control, it may be possible to signal to the congestion control mechanism that there was congestion (loss) even when a packet has been recovered, e.g. using ECN, to reduce the impact on the non-coded flows (see Section 5.2 and [TENTET]).¶
The congestion control may not be aware of the existence of a coding scheme underneath it. The congestion control may behave as if no coding scheme had been introduced. The only way for a coding channel to indicate that symbols have been lost but recovered is to exploit existing signaling that is understood by the congestion control mechanism. An example would be to indicate to a TCP sender that a packet has been received, yet congestion has occurred, by using ECN signaling [TENTET].¶
The coding rate can be tuned depending on the number of recovered symbols and the rate at which the sender transmits data. If the coding scheme is not aware of the congestion control implementation, it is hard for the coding scheme to apply the relevant coding rate.¶
Useless repair symbols only impact the load on the network without actual gain for the coded flow. Using feedback signaling, FEC mechanisms can measure the ratio between the number of symbols that were actually used and the number of symbols that useless, and adjust the coding rate.¶
The transport above the FEC channel may support out-of-order delivery of packets: reordering mechanisms at the receiver may not be necessary. In cases where the transport requires in-order delivery, the FEC channel may need to implement a reordering mechanism. Otherwise, spurious retransmissions may occur at the transport level.¶
The transport or application layer above the FEC channel may require partial reliability only. FEC may provide an unnecessary service unless it is aware of the reliability requirements. Partial reliability impacts the type of FEC and type of codec that can be used, such as discussed in Section 2.5.¶
The transport may exploit multiple paths without the FEC channel being aware of it. If FEC is aware that multiple paths are in use, FEC can be applied to all subflows as an aggregate, or to each of the subflows individually. If FEC is not aware that multiple paths are in use, FEC can only be applied to each subflow individually. When FEC is applied to all the flows as an aggregate, the varying characteristics of the individual paths may lead to a risk for the coding rate to be inadequate for the characteristics of the individual paths.¶
This section provides a short state-of-the art overview of activities related to congestion control and coding. The objective is to identify open research questions and contribute to advice when evaluating coding mechanisms.¶
We map activities related to congestion control and coding with the organization presented in this document:¶
There is a general trade-off, inherent to the use of coding, between (1) reducing goodput when useless repair symbols are transmitted and (2) helping to recover from transmission and congestion losses.¶
There is a trade-off related to the amount of redundancy to add, as a function of the transport layer protocol and application requirements.¶
[RFC8095] describes the mechanisms provided by existing IETF protocols such as TCP, SCTP or RTP. [RFC8406] describes the variety of coding techniques. The number of combinations makes the determination of an optimum parameters derivation very complex. This depends on application requirements and deployment context.¶
Appendix C of [RFC8681] describes how to tune the parameters for target use-case. However, this discussion does not integrate congestion-controlled end points.¶
Research question 1 : "Is there a way to dynamically adjust the codec characteristics depending on the transmission channel, the transport protocol and application requirements ?"¶
Research question 2 : "Should we apply specific per-stream FEC mechanisms when multiple streams with different reliability needs are carried out ?"¶
Recovering lost symbols may hide congestion losses from the congestion control. Disambiguate acked packets from rebuilt packets would help the sender adapt its sending rate accordingly. There are opportunities for introducing interaction between congestion control and coding schemes to improve the quality of experience while guaranteeing fairness with other flows.¶
Some existing solutions already propose to disambiguate acked packets from rebuilt packets [QUIC-FEC]. New signaling methods and FEC-recovery-aware congestion controls could be proposed. This would allow the design of adaptive coding rates.¶
Research question 3 : "Should we quantify the harm that a coded flow would induce on a non-coded flow ? How can this be reduced while still benefiting from advantages brought by FEC ?"¶
Research question 4 : "If transport and FEC senders are collocated and close to the client, and FEC is applied only on the last mile, e.g. to ignore losses on a noisy wireless link, would this raise fairness issues ?"¶
Research question 5 : "Should we propose a generic API to allow dynamic interactions between a transport protocol and a coding scheme ? This should consider existing APIs between application and transport layers."¶
Research Recommendation 1: "From a congestion control point-of-view, a recovered packet must be considered as a lost packet. This does not apply to the usage of FEC on a path that is known to be lossy."¶
Research Recommendation 2: "New research contributions should be mapped following the organization of this document (above, below, in the congestion control) and should consider congestion control aspects when proposing and comparing FEC coding solutions in communication systems."¶
Research Recommendation 3: "When a research work aims at improving throughput by hiding the packet loss signal from congestion control (e.g., because the path between the sender and receiver is known to consist of a noisy wireless link), the authors should 1) discuss the advantages of using the proposed FEC solution compared to replacing the congestion control by one that ignores a portion of the encountered losses, 2) critically discuss the impact of hiding packet loss from the congestion control mechanism."¶
Many thanks to Spencer Dawkins, Dave Oran, Carsten Bormann, Vincent Roca and Marie-Jose Montpetit for their useful comments that helped improve the document.¶
This memo includes no request to IANA.¶
FEC and CC schemes can contribute to DoS attacks. Moreover, the transmission of signaling messages from the client to the server should be protected and reliable otherwise an attacker may compromise FEC rate adaptation. Indeed, an attacker could either modify the values indicated by the client or drop signaling messages.¶
In case of FEC below the transport, the aggregate rate of source and repair packets may exceed the rate at which a congestion control mechanism allows an application to send. This could result in an application obtaining more than its fair share of the network capacity.¶