| Internet-Draft | QuicR Media | March 2023 |
| Jennings & Nandakumar | Expires 14 September 2023 | [Page] |
This specification QuicR, an unified media delivery protocol over QUIC. It aims at supporting multiple application classes with varying latency requirements including ultra low latency applications such as interactive communication and gaming. It is based on a publish/subscribe metaphor where entities publish and subscribe to data that is sent through, and received from, relays in the cloud. The information subscribed to is named such that this forms an overlay information centric network. The relays allow for efficient large scale deployments.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 14 September 2023.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
This specification defines QuicR, a publish and subscribe based media delivery protocol over QUIC, where the principal idea is entities publish unique named objects that are end-to-end encrypted and consume data by subscribing to the named objects. The names (Section 4) used in the QuicR protocol are scoped and authorized to the domain operating the application server (referred to as Origin in this specification). The authorization is scoped to a QuicR Namespace (Section 4.2) that identify a range of Names that share a common prefix with the Namespace.¶
The published data carry metadata identifying relative priority, time-to-live and other useful metadata that's authenticated for components implementing Relay functions to make drop/forwarding decisions. QuicR is designed to make it easy to implement relays so that fail over could happen between relays with minimal impact to the clients and relays can redirect a client to a different relay.¶
QuicR provides services based on application requirements (with the support of underlying transport, where necessary) such as estimation of available bandwidth, fragmentation and reassembly, resiliency, congestion control and prioritization of data delivery based on data lifetime and importance of data. It is designed to be NAT and firewall traversal friendly and can be fronted with load balancers.¶
┌───────────┐ ┌───────────┐ ┌─────────┐
│ │ │ │ │ │
│ Publisher │ │ Relay │ │ Origin │
│ │ │ │ │ │
└───────────┘ └───────────┘ └─────────┘
│ │ │
│ │ │
│┌──────────────────────────┴─────────────────────────────┐│
││Session Login - exchange configuration - tokens, names..││
│└──────────────────────────┬─────────────────────────────┘│
│ │ │
│ │ │
│ publish_intent(.., │ │
│ token) │ │
│ │ │
├──────────────────────────▶│◉◉◉◉ Validate │
│ │ ◉ token │
│ │◉◉◉◉ │
│ │ │
│ │ │
│ │ publish_intent(.., │
│ │ token) │
│ │ │
│ │─────────────────────────────▶│ ◉◉◉◉
│ │ │ ◉ Validate Request
│ │ │ ◉◉◉◉
│ │ │
│ │ publish_intent_ok(..) │
│ │ │
│ ◀──────────────────────────────┤
│ │ │
│ │ │
│ │◉◉◉◉ Save namespace, │
│ │ ◉ publisher │
│ publish_intent_ok(..) │◉◉◉◉ │
│ │ │
│◀──────────────────────────┤ │
│ │ │
│ │ │
│ ┌─────────────────────────┴───────────────────────────┐ │
│ │ Publish media objects │ │
│ └─────────────────────────┬───────────────────────────┘ │
│ │ │
¶
┌────────────┐ ┌───────────┐ ┌─────────┐
│ │ │ │ │ │
│ Subscriber │ │ Relay │ │ Origin │
│ │ │ │ │ │
└─────┬──────┘ └───────────┘ └─────────┘
│ │ │
│ │ │
│┌──────────────────────────┴─────────────────────────────┐│
││Session Login - exchange configuration - tokens, names..││
│└──────────────────────────┬─────────────────────────────┘│
│ │ │
│ │ │
│ subscribe(.., token) │ │
│ │ │
│ │ │
├──────────────────────────▶│◉◉◉◉ Validate │
│ │ ◉ token │
│ │◉◉◉◉ │
│ │ │
│ │ │
│ │ subscribe(.., token) │
│ │ │
│ │ │
│ │─────────────────────────────▶│ ◉◉◉◉
│ │ │ ◉ Validate Request
│ │ │ ◉◉◉◉
│ │ │
│ │ subscribe_ok(..) │
│ │ │
│ ◀──────────────────────────────┤
│ │ │
│ │ │
│ │◉◉◉◉ Save subscriber │
│ │ ◉ info │
│ subscribe_ok(..) │◉◉◉◉ │
│ │ │
│◀──────────────────────────┤ │
│ │ │
│ │ │
│ ┌─────────────────────────┴───────────────────────────┐ │
│ │ Media objects will be delivered matching the sub │ │
│ └─────────────────────────┬───────────────────────────┘ │
│ │ │
¶
TODO¶
QuicR Name/Name: Identifier associated with the published object within the QuicR Delivery Network. QuicR Names are globally unique and are scoped to publishers.¶
QuicR Namespace/Namespace: A QuicR Namespace is a partial representation for a given QuicR Name. The namespace identifies a range of names the share the common prefix and provides context for authorization for publishing and subscribing to the names under a given Namespace.¶
Named (Media/Data) Object: A named object is an application level chunk of Data that is identified by its QuicR Name. A media object thus has a unique Name, a lifetime, priority and carries end-to-end encrypted application data, along with other things and is transported via the QuicR protocol.¶
Relay Function: Functionality of the QuicR architecture, that implements store and forward behavior at the minimum. Such a function typically receives subscriptions and publishes data to the other endpoints that have subscribed to the named data. Such functions may cache the data as well for optimizing the delivery experience.¶
Relay: Server component (physical/logical) in the cloud that implements the Relay Function.¶
Publisher: An endpoint that sends named objects to a Relay. [ also referred to as producer of the named object]¶
Subscriber: An endpoint that subscribes and receives the named objects. Relays can act as subscribers to other relays. Subscribers can also be referred to as consumers.¶
Client/QuicR Client: An endpoint that acts as a Publisher, Subscriber, or both. May also implement a Relay Function in certain contexts.¶
Named Object: Application level chunk of Data that has a unique Name, a limited lifetime, priority and is transported via the protocol defined in this specification.¶
Origin server: Component managing/authoring the names scoped under a domain for a specific application and is responsible for establishing trust between clients and relays for delivering media. Origin servers MAY implement other QuicR functions, such as Relay function, as necessary.¶
Control Stream: QUIC Stream to exchange control message to setup appropriate context for media delivery and is scoped to a given QUIC Connection. Control channels are scoped to a given QUIC connection. Functionally, Control Messages enable authorization of names, setting up media properties and starting/terminating media sessions.¶
Media Stream/Data Stream: QUIC Stream or QUIC Datagram based transport for delivering end to end encrypted application media objects. Such objects shall carry metadata (unencrypted) for Relays to make store/forwarding decisions along with the application payload.¶
Names are basic elements with in the QuicR architecture and they uniquely identify objects. For publishers of the media, the names identify application defined data objects being contributed and for the subscribers/receivers, the names correspond to application data objects to be consumed.¶
The scope and granularity of the names and the data objects they represent are application defined and controlled.¶
However, a given QuicR name must maintain certain properties as given below¶
Each published name must be unique and is scoped to a given domain and an application under that domain.¶
Names support a way for the subscribers to request for the associated data either by specifying the full or partial names. The latter is supported via wildcarding.¶
Named objects should enable caching in relays in a way CDNs cache resources and thus can obtain similar benefits such caching mechanisms would offer.¶
The names of each object in QuicR is composed of the following components:¶
The template identifier forms the first part of a given QuicR Name. It identifies the domain hosting a given application and the context information identifying the application under that domain. The Domain component is like a HTTP Origin or an standardized identifier (similar to IANA PEN) and the application context provides necessary information to uniquely identify the application under the domain.¶
Data identifier further aspects of application that uniquely defines the data being published, for example video stream from a conference user or a message in a chat room for a messaging application. The breakdown of a given data identifier is application specifc and controlled.¶
Example: In the example below the template identifier indicates the the domain as acme.meeting.com with the application context identifying instance of a meeting under this domain, say "meeting123". The data component captures a QuicR named object corresponding to a high resolution camera stream from the user "alice" being published as object 17 under group 17.¶
Example 1 quicr://acme.meeting.com/v1/meeting123/alice/cam5/HiRes/15/17¶
Example 2 quicr://messages.com/v1/orgAbc/spaces/¶
Once a named object is created, the content inside the named object can never be changed. Objects have an expiry time after which they should be discarded by caches. Objects have an priority that the relays and clients can use to make drop decisions or sequencing the sending order. The data inside an object is end-to-end encrypted whose keys are not available to Relay(s).¶
QuicR supports the idea of QuicR Namespaces, that enables aggregation of
objects with a common prefix. Wildcard names are formed by skipping the
right most segments of the name and is represented in the form of x/y,where
x is the name where only first y bits are used and rest are wildcard
and matches any name where the first y bits match. When converting to a URI,
the URI for such names end with a *.¶
Thus a QuicR Namespace covers a range of names matching a common prefix and is used:¶
a) By Publishers to express their intent to publish a range of names
b) By Subscribers to request for named objects to be made as aggregates instead of
requests made at the object level granularity,
¶
In both the cases above the QuicR Namespace provides the necessary authorization context.¶
For example, in a web conferencing use case, the client may subscribe to just the meeting_id and one of the publishers so as to get all the media from that user in a particular. The example matches all the named objects published by the user alice in the meeting123.¶
quicr://acme.meeting.com/meeting123/alice/*¶
Objects within QuicR belong to a group. A group (a.k.a group of objects) represent an independent composition of set of objects, where there exists dependency relationship between the objects within the group. Groups, thus can be independently consumable by the subscriber applications. Groups also represent sync/transition points enabling subscribers to consume at right point in time and for caches to apply appropriate caching and congestion strategies.¶
QuicR supports delivering media over QUIC Streams as well as over QUIC Datagrams as chosen by the application.¶
Media delivery in QuicR is started by the publisher/subscriber setting up a "Control Stream" for one or more QuicR Namespaces. The control stream, which is based on QUIC stream, is used to configure and setup properties for the "Data Stream". Named data is delivered over the Data Stream over QUIC streams or QUIC datagrams based on the application settings. The Control Channel can also be used to configure in-session parameters.¶
When a client or relay begins a transaction with the relay/origin, the client starts by opening a new bilateral stream. This stream will act as the "control stream" for the exchange of data, carrying a series of control messages in both directions.¶
The control stream will remain open as long as the peers are still sending or receiving the media. If either peer closes the control stream, the other peer will close its end of the stream and discard the state associated with the media transfer.¶
Streams are "one way". If a peer both sends and receive media, there will be different control streams for sending and receiving.¶
The control channel carry series of messages, encoded as a length followed by a message value:¶
quicr_message {
length(16),
value(...)
}
¶
The length is encoded as a 16 bit number in big endian network order.¶
Entities that intend to receive named objects will do so via subscriptions to the named objects.¶
enum subscribe_intent
{
immediate(0),
catch_up(1),
wait_up(2),
}
quicr_subscribe_message {
* message_type(i),
* namespace_length(i),
* namespace(...),
* subscribe_intent intent,
* auth_info_length(i),
* auth_info(...)
}
¶
The message type will be set to SUBSCRIBE. namespace identifies the
QuicR Namespace and auth_info carries authorization information.¶
The intent field specifies how the Relay Function should provided the
named objects to the client. Following options are defined for
the intent¶
immediate: Deliver any new objects it receives that match the namespace.¶
catch_up: Deliver any new objects it receives and in addition send any previous objects it has received that matches the namespace, under the current group.¶
wait_up: Wait until next sync point (group) before delivering the objects.¶
Subscriptions are typically long-lived transactions and they stay active until one of the following happens¶
While the subscription is active for a given name, the Relay(s)
must send named objects it receives to all the matching subscribers.
A QuicR client can renew its subscriptions at any point by sending a
new quicr_subscribe_message. Such subscriptions
MUST refresh the existing subscriptions for that name.¶
TODO: provide more details on authorization flows.¶
Subscriptions are aggregated at entities that perform Relay Function. Aggregating subscriptions helps reduce the number of subscriptions for a given named object in transit and also enables efficient distribution of published media with minimal copies between the client and the origin server/ or other relays, as well as reduce the latencies when there are multiple subscribers for a given namespace object behind a given cloud server.¶
A quicr_subscribe_reply provides result of the subscription.¶
enum response
{
ok(0),
expired(1)
fail(2)
}
quicr_subscribe_reply
{
Response response,
[Reason Phrase Length (i),
[Reason Phrase (..)],
}
¶
A response of ok indicates successful subscription, for failed
or expired responses, "Reason Phrase" shall be populated
with appropriate reason.¶
The quicr_publish_intent sets up authorization for one or more
QuicR Namespaces that the publisher intends to publish data with.
The authorization is carried out based on authorization token obtained
via out of band mechanisms. Either the Relay at the edge or the Origin Server
may validate the token depending on the configuration.¶
quicr_publish_intent {
* message_type(i),
* namespace_length(i),
* namespace(...)...,
* auth_info_length(i),
* auth_info(...)
* }
¶
The message type will be set to PUBLISH_INTENT (6). namespace identifies the
QuicR Namespace and auth_info carries authorization token.¶
TODO: provide more details on authorization flows.¶
quicr_publish_intent_reply provides the result of intent to publish on a namespace.¶
quicr_publish_intent_reply {
* message_type(i),
* Response response,
* [Reason Phrase Length (i),
* [Reason Phrase (..)]
}
¶
The message id is set to PUBLISH_INTENT_OK (7).¶
This message enables cloud relays to know the authorized names from a given Publisher. This helps to make caching decisions, deal with collisions and so on.¶
A>A cloud relay could start caching the data associated with the names that has
not been validated yet by the origin server and decide to flush its cache
if no response to the publish_intent is received within a given implementation
defined timeout. This is an optimization that would allow publishers to start
transmitting the data without needing to wait a RTT.¶
The quicr_fragment_message message is used to convey the content of a media stream as a series of fragments. This message is sent when sending over QUIC Streams.¶
quicr_fragment_message {
* message_type(i),
* name_length(i),
* name(...)...,
* best_before(i),
* flags(i),
* offset_and_fin(i),
* length(i),
* data(...)
}
flags := Reserved (3) | IsDiscardable (1) | Priority (3)
¶
The message type will be set to FRAGMENT (5). The offset_and_fin field encodes
two values, as in:¶
offset_and_fin = 2 * offset + is_last_fragment¶
The flag is_last_fragment is set to 1 if this fragment is the last one
of an object.¶
quicr_relay_redirect_message enables relay failover scenarios that is sent
in response to PUBLISH, PUBLISH_INTENT and SUBSCRIBE messages indicating
the new relay to the clients.¶
quicr_relay_redirect_message
{
* message_type(i),
* relay_address_length(i),
* relay_address(...)
}
¶
If transmission as datagram is preferred, the media fragments are sent as QUIC Datagram frames.¶
The datagram frames are encoded as a datagram header, followed by the bytes in the fragment:¶
datagram_frame_content {
datagram_header,
datagram_content
}
¶
The datagram header is defined as:¶
* quicr_datagram_header {
* name_length (i),
* name(...)...,
* offset_and_fin (i),
* best_before(i),
* flags (8),
* }
flags := Reserved (3) | IsDiscardable (1) | Priority (3)
¶
The offset_and_fin field encodes two values, as in:¶
offset_and_fin = 2*offset + is_last_fragment¶
The Relays receive subscriptions and intent to publish request and optionaly forward them towards the origin for authorization. Subscriptions received are aggregated. When a relay receives a publish request with data, it will forward it both towards the Origin and to any clients or relays that have a matching subscription. This "short circuit" of distribution by a relay before the data has even reached the Origin servers provides significant latency reduction for nearby client. Relays MAY cache some of the information for short period of time and the time cached may depend on the Origin. The Relay keeps an outgoing queue of objects to be sent to the each subscriber and objects are sent in priority order.¶
At a high level, Relay Function within QuicR architecture support store and forward behavior. Relay function can be realized in any component of the QuicR architecture depending on the application. Typical use-cases might require the intermediate servers (caches) and the origin server to implement the relay function. However the endpoint themselves can implement the Relay function in a Isomorphic deployment, if needed.¶
The relays are capable of receiving data in stream mode or in datagram mode. In both modes, relays will cache and deliver fragments as they arrive.¶
Relays makes use of priority, time-to-live, is_discardable metadata properties from the published data to make forward or drop decisions when reacting to congestion as indicated by the underlying QUIC stack. The same can be used to make caching decisions.¶
Relays store objects no more than best_before time associated with the
object. Congestion/Rate control feedback can further influence what
gets cached based on the relative priority and rate at which data
can be delivered. Local cache policies can also limit the amount and
duration of data that can be cached.¶
A relay that wants to shutdown shall use the redirect message to move traffic to a new relay. If a relay has failed and restarted or been load balanced to a different relay, the client will need to resubscribe to the new relay after setting up the connection.¶
TODO: Cluster so high reliable relays should share subscription info and publication to minimize of loss of data during a full over.¶
TODO¶
Thanks to TODO for contributions and suggestions to this specification.¶