TCPLS: Modern Transport Services with TCP and TLS

4.2.1. Joining TCP connections

The TCPLS server provides tokens to the client in order to join new TCP connections to the TCPLS session. Figure 7 illustrates a client and server first establishing a new TCPLS session as described in Section 4. Then the server sends a token over this connection using the New Token frame. Each token has a sequence number (e.g. 1) and a value (e.g. "abc"). The client uses this token to open a new TCP connection and initiates the TCPLS handshake. It adds the token inside the TCPLS Join TLS extension in the ClientHello.¶

             <-1.TCPLS Handshake->
       .---------------------------------.
       |            <-2.New Token(1,abc) |
       v                                 v
+--------+                            +--------+
| Client |                            | Server |
+--------+                            +--------+
       ^                                 ^
       | 3.TCPLS Handshake + Join(abc)-> |      Legend:
       .---------------------------------.        --- TCP connection

When receiving a TCPLS Join Extension, the server validates the token and associates the TCP connection to the TCPLS session.¶

Each TCP connection that is part of a TCPLS session is identified by a 32-bit unsigned integer called its Connection ID. The first TCP connection of a session corresponds to Connection ID 0. When joining a new connection, the sequence number of the token, i.e. 1 in our example, becomes the Connection ID of the connection. The Connection ID enables the Client and the Server to identify a specific TCP connection within a given TCPLS session.¶

4.2.2. Failover

TCPLS supports two types of failover. In make-before-break, the client creates a TCP connection using the procedure described in Section 4.2.1 but only uses it once the initial connection fails.¶

In break-before-make, the client creates the initial TCP connection and uses it for the TCPLS handshake and the data. The server advertises one or more tokens over this connection. Upon failure of the initial TCP connection, the client initiates a second TCP connection using the server-provided token.¶

In both cases, some records sent by the client or the server might be in transit when the failure occurs. Some of these records could have been partially received but not yet delivered to the TCPLS layer when the underlying TCP connection fails. Other records could have already been received, decrypted and data of their frames could have been delivered to the application. To prevent data losses and duplication, TCPLS includes its own acknowledgments.¶

A TCPLS receiver acknowledges the received records using the ACK frame. Records are acknowledged after the record protection has been successfully removed. This enables the sender to know which records have been received. TCPLS enables the endpoint to send acknowledgments for a TCP connection over any connections, e.g. not only the receiving connection.¶

4.2.3. Migration

To migrate from a given TCP connection, an endpoint stops transmitting over this TCP connection and sends the following frames on other TCP connections. It leverages the acknowledgments to retransmit the frames of TLS records that have not been yet acknowledged.¶

When an endpoint abortfully closes a TCP connection, its peer leverages the acknowlegments to retransmit the TLS records that were not acknowlegded.¶

4.2.4. Multipath

TCPLS also supports the utilization of different TCP connections, over different paths or interfaces, to improve throughput or spread stream frames over different TCP connections. When the endpoints have opened several TCP connections, they can send frames over the connections. TCPLS can send all the stream frames belonging to a given stream over one or more underlying TCP connections. The latter enables bandwidth aggregation by using TCP connections established over different network paths.¶

5.1.1. TCPLS

The "tcpls" extension is used by the client and the server to announce their support of TCPLS. The extension contains no value. When it is present in both the ClientHello and the EncryptedExtensions, the endpoints MUST use TCPLS after completing the TLS handshake.¶

5.1.2. TCPLS Join

struct {
    opaque token<32>;
} Join;

The "tcpls_join" extension is used by the client to join the TCP connection on which it is sent to a TCPLS session. The extension contains a Token provided by the server. The client MUST NOT send more than one "tcpls_join" extension in its ClientHello. When receiving a ClientHello with this extension, the server checks that the token is valid and joins the TCP connection to the corresponding TCPLS session. When the token is not valid, the server MUST abort the handshake with an illegal_parameter alert.¶

By controlling the amount of tokens given to the client, the server can control the number of active TCP connections of a TCPLS session. The server SHOULD replenish the tokens when TCP connections are removed from the TCPLS session.¶

5.2.1. Padding frame

This frame has no semantic value. It can be used to mitigate traffic analysis on the TLS records of a TCPLS session. The Padding frame has no content.¶

Padding frame {
    Type (8) = 0x00,
}

5.2.2. Ping frame

This frame is used to elicit an acknowledgment from its peer. It has no content. When an endpoint receives a Ping frame, it acknowledges the TLS record that contains this frame. This frame can be used by an endpoint to check that its peer can receive TLS records over a particular TCP connection.¶

Ping frame {
    Type (8) = 0x01,
}

5.2.3. Stream frame

This frame is used to carry chunks of data of a given stream.¶

Stream frame {
    Type (7) = 0x01,
    FIN (1),
    Stream ID (32),
    Offset (64),
    Length (16),
    Stream Data (...),
}

FIN:

The last bit of the frame type bit indicates that this Stream frame ends the stream when its value is 1. The last byte of the stream is at the sum of the Offset and Length fields of this frame.¶

Stream ID:

A 32-bit unsigned integer indicating the ID of the stream this frame relates to.¶

Offset:

A 64-bit unsigned integer indicating the offset in bytes of the carried data in the stream.¶

Length:

A 16-bit unsigned integer indicating the length of the Stream Data field.¶

5.2.4. ACK frame

This frame is sent by the receiver to acknowledge the receipt of TLS records on a particular TCP connection of the TCPLS session. Although the reliability of the data exchange on a connection is handled by TCP, there are situations such as the failure of a TCP connection where a sender does not know whether the TLS frames that it sent have been correctly received by the peer. The ACK frame allows a TCPLS receiver to indicate the highest TLS record sequence number received on a specific connection. The ACK frame can be sent over any TCP connection of a TCPLS session.¶

ACK frame {
    Type (8) = 0x04,
    Connection ID (32),
    Highest Record Sequence Received (64),
}

Connection ID:

A 32-bit unsigned integer indicating the TCP connection for which the acknowledgment was sent.¶

Highest Record Sequence Received:

A 64-bit unsigned integer indicating the highest TLS record sequence number received on the connection indicated by the Connection ID.¶

5.2.5. New Token frame

This frame is used by the server to provide tokens to the client. Each token can be used to join a new TCP connection to the TCPLS session, as described in Section 4.2.1. Clients MUST NOT send New Token frames.¶

New Token frame {
    Type (8) = 0x05,
    Sequence (8),
    Token (256),
}

Sequence:

A 8-bit unsigned integer indicating the sequence number of this token¶

Token:

A 32-byte opaque value that can be used as a token by the client.¶

5.2.6. Connection Reset frame

This frame is used by the receiver to inform the sender that a TCP connection has been reset.¶

Connection Reset frame {
    Type (8) = 0x06,
    Connection ID (32)
}

Connection ID:

A 32-bit unsigned integer indicating the ID of the connection that failed.¶

5.2.7. New Address frame

This frame is used by an endpoint to add a new local address to the TCPLS session. This address can then be used to establish new TCP connections. The server advertises addresses that the client can use as destination when adding TCP connections. The client advertises address that it can use as source when adding TCP connections.¶

New Address frame {
    Type (8) = 0x07,
    Address ID (8),
    Address Version (8),
    Address (..),
    Port (16),
}

Address ID:

A 8-bit identifier for this address. For a given Address ID, an endpoint receiving a frame with a content that differs from previously received frames MUST ignore the frame. An endpoint receiving a frame for an Address ID that was previously removed MUST ignore the frame.¶

Address Version:

A 8-bit value identifying the Internet address version of this address. The number 4 indicates IPv4 while 6 indicates IPv6.¶

Address:

The address value. Its size depends on its version. IPv4 addresses are 32-bit long while IPv6 addresses are 128-bit long.¶

Port:

A 16-bit value indicating the TCP port used with this address.¶

5.2.8. Remove Address frame

This frame is used by an endpoint to announce that it is not willing to use a given address to establish new TCP connections. After receiving this frame, a client MUST NOT establish new TCP connections to the given address. After receiving this frame, an endpoint MUST close all TCP connections using the given address.¶

Remove Address frame {
    Type (8) = 0x08,
    Address ID (8),
}

Address ID:

A 8-bit identifier for the address to remove. An endpoint receiving a frame for an address that was nonexistent or already removed MUST ignore the frame.¶