Internet-Draft | Oblivious HTTP | January 2021 |
Thomson & Wood | Expires 1 August 2021 | [Page] |
This document describes a system for the forwarding of encrypted HTTP messages. This allows clients to make requests of servers without the server being able to link requests to other requests from the same client.¶
This note is to be removed before publishing as an RFC.¶
Discussion of this document takes place on the HTTP Working Group mailing list (http@ietf.org), which is archived at https://mailarchive.ietf.org/arch/browse/http/.¶
Source for this draft and an issue tracker can be found at https://github.com/unicorn-wg/oblivious-http.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 1 August 2021.¶
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶
The act of making a request using HTTP reveals information about the client identity to a server. Though the content of requests might reveal information, that is information under the control of the client. In comparison, the source address on the connection reveals information that a client has only limited control over.¶
Even where an IP address is not directly attributed to an individual, the use of an address over time can be used to correlate requests. Servers are able to use this information to assemble profiles of client behavior, from which they can make inferences about the people involved. The use of persistent connections to make multiple requests improves performance, but provides servers with additional certainty about the identity of clients in a similar fashion.¶
Use of an HTTP proxy can provide a degree of protection against servers correlating requests. Systems like virtual private networks or the Tor network [Dingledine2004], provide other options for clients.¶
Though the overhead imposed by these methods varies, the cost for each request is significant. Preventing request linkability requires that each request use a completely new TLS connection to the server. At a minimum, this requires an additional round trip to the server in addition to that required by the request. In addition to having high latency, there are significant secondary costs, both in terms of the number of additional bytes exchanged and the CPU cost of cryptographic computations.¶
This document describes a method of encapsulation for binary HTTP messages [BINARY] using Hybrid Public Key Encryption (HPKE; [HPKE]). This protects the content of both requests and responses and enables a deployment architecture that can separate the identity of a requester from the request.¶
Though this scheme requires that servers and proxies explicitly support it, this design represents a performance improvement over options that perform just one request in each connection. With limited trust placed in the proxy (see Section 8), clients are assured that requests are not uniquely attributed to them or linked to other requests.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
An HTTP request that is encapsulated in an HPKE-encrypted message; see Section 5.1.¶
An HTTP response that is encapsulated in an HPKE-encrypted message; see Section 5.2.¶
An intermediary that forwards requests and responses between clients and a single oblivious request resource.¶
A resource that can receive an encapsulated request, extract the contents of that request, forward it to an oblivious target resource, receive a response, encapsulate that response, then return that response.¶
The resource that is the target of an encapsulated request. This resource logically handles only regular HTTP requests and responses and so might be ignorant of the use of oblivious HTTP to reach it.¶
This draft includes pseudocode that uses the functions and conventions defined in [HPKE].¶
Encoding an integer to a sequence of bytes in network byte order is described
using the function encode(n, v)
, where n
is the number of bytes and v
is
the integer value. The function len()
returns the length of a sequence of
bytes.¶
Formats are described using notation from Section 1.3 of [QUIC].¶
A client learns the following:¶
This information allows the client to make a request of an oblivious target resource without that resource having only a limited ability to correlate that request with the client IP or other requests that the client might make to that server.¶
In order to make a request to an oblivious target resource, the following steps occur, as shown in Figure 1:¶
A client needs to acquire information about the key configuration of the oblivious request resource in order to send encapsulated requests.¶
In order to ensure that clients do not encapsulate messages that other entities can intercept, the key configuration MUST be authenticated and have integrity protection. One way to ensure integrity for key configuration is for the oblivious request resource to serve content to the client directly, using HTTPS and the "application/ohttp-keys" media type; see Section 4.2.¶
Specifying a format for expressing the information a client needs to construct an encapsulated request ensures that different client implementations can be configured in the same way. This also enables advertising key configurations in a consistent format.¶
A client might have multiple key configurations to select from when encapsulating a request. Clients are responsible for selecting a preferred key configuration from those it supports. Clients need to consider both the key encapsulation method (KEM) and the combinations of key derivation function (KDF) and authenticated encryption with associated data (AEAD) in this decision.¶
Evolution of the key configuration format is supported through the definition of new formats that are identified by new media types.¶
A single key configuration consists of a key identifier, a public key, an identifier for the KEM that the public key uses, and a set HPKE symmetric algorithms. Each symmetric algorithm consists of an identifier for a KDF and an identifier for an AEAD.¶
Figure 2 shows a single key configuration, KeyConfig, that is expressed using the TLS syntax; see Section 3 of [TLS].¶
The types HpkeKemId, HpkeKdfId, and HpkeAeadId identify a KEM, KDF, and AEAD respectively. The definitions for these identifiers and the semantics of the algorithms they identify can be found in [HPKE].¶
The "application/ohttp-keys" format is a media type that identifies a serialized collection of key configurations. The content of this media type comprises one or more key configuration encodings (see Section 4.1) that are concatenated.¶
application¶
ohttp-keys¶
N/A¶
None¶
only "8bit" or "binary" is permitted¶
N/A¶
this specification¶
N/A¶
N/A¶
see Authors' Addresses section¶
COMMON¶
N/A¶
see Authors' Addresses section¶
IESG¶
HTTP message encapsulation uses HPKE for request and response encryption. An encapsulated HTTP message includes the following values:¶
The encoding of an HTTP message is as follows:¶
Plaintext Message { Message Length (i), Message (..), Padding Length (i), Padding (..), }¶
An Encapsulated Request is comprised of a length-prefixed key identifier and a
HPKE-protected request message. HPKE protection includes an encapsulated KEM
shared secret (or enc
), plus the AEAD-protected request message. An
Encapsulated Request is shown in Figure 3. Section 5.1 describes the
process for constructing and processing an Encapsulated Request.¶
Responses are bound to responses and so consist only of AEAD-protected content. Section 5.2 describes the process for constructing and processing an Encapsulated Response.¶
The size of the Nonce field in an Encapsulated Response corresponds to the size of an AEAD key for the corresponding HPKE ciphersuite.¶
Clients encapsulate a request request
using values from a key configuration:¶
keyID
,¶
pkR
, and¶
kdfID
, and AEAD, identified by
aeadID
.¶
The client then constructs an encapsulated request, enc_request
, as follows:¶
pkR
, yielding context
and encapsulation
key enc
.¶
aad
, by concatenating the values of keyID
,
kdfID
, and aeadID
, as 8-, 16- and 16-bit integers respectively, each in
network byte order.¶
request
with aad
as associated data using context
,
yielding ciphertext ct
.¶
aad
, enc
, and ct
, yielding an Encapsulated
Request enc_request
.¶
Note that enc
is of fixed-length, so there is no ambiguity in parsing this
structure.¶
In pseudocode, this procedure is as follows:¶
enc, context = SetupBaseS(pkR, "request") aad = concat(encode(1, keyID), encode(2, kdfID), encode(2, aeadID)) ct = context.Seal(aad, request) enc_request = concat(aad, enc, ct)¶
Servers decrypt an Encapsulated Request by reversing this process. Given an
Encapsulated Request enc_request
, a server:¶
Parses enc_request
into keyID
, kdfID
, aeadID
, enc
, and ct
(indicated using the function parse()
in pseudocode). The server is then
able to find the HPKE private key, skR
, corresponding to keyID
.¶
a. If keyID
does not identify a key, the server returns an error.¶
b. If kdfID
and aeadID
identify a combination of KDF and AEAD that the
server is unwilling to use with skR
, the server returns an error.¶
skR
and the encapsulated key enc
,
yielding context
.¶
aad
, from keyID
, kdfID
, and
aeadID
or as the first five bytes of enc_request
.¶
ct
using aad
as associated data, yielding request
or an error
on failure. If decryption fails, the server returns an error.¶
In pseudocode, this procedure is as follows:¶
keyID, kdfID, aeadID, enc, ct = parse(enc_request) aad = concat(encode(1, keyID), encode(2, kdfID), encode(2, aeadID)) context = SetupBaseR(enc, skR, "request") request, error = context.Open(aad, ct)¶
Given an HPKE context context
, a request message request
, and a response
response
, servers generate an Encapsulated Response enc_response
as
follows:¶
secret
from context
, using the string "response" as context.
The length of this secret is max(Nn, Nk)
, where Nn
and Nk
are the length
of AEAD key and nonce associated with context
.¶
max(Nn, Nk)
bytes, called response_nonce
.¶
prk
using the Extract
function provided by
the KDF algorithm associated with context
. The ikm
input to this
function is secret
; the salt
input is the concatenation of enc
(from
enc_request
) and response_nonce
¶
Expand
function provided by the same KDF to extract an AEAD key
key
, of length Nk
- the length of the keys used by the AEAD associated
with context
. Generating key
uses a label of "key".¶
Expand
function to extract a nonce nonce
of length Nn
-
the length of the nonce used by the AEAD. Generating nonce
uses a label of
"nonce".¶
response
, passing the AEAD function Seal the values of key
,
nonce
, empty aad
, and a pt
input of request
, which yields ct
.¶
response_nonce
and ct
, yielding an Encapsulated Response
enc_response
. Note that response_nonce
is of fixed-length, so there is no
ambiguity in parsing either response_nonce
or ct
.¶
In pseudocode, this procedure is as follows:¶
secret = context.Export("response", Nk) response_nonce = random(max(Nn, Nk)) salt = concat(enc, response_nonce) prk = Extract(salt, secret) aead_key = Expand(secret, "key", Nk) aead_nonce = Expand(secret, "nonce", Nn) ct = Seal(aead_key, aead_nonce, "", response) enc_response = concat(response_nonce, ct)¶
Clients decrypt an Encapsulated Request by reversing this process. That is,
they first parse enc_response
into response_nonce
and ct
. They then
follow the same process to derive values for aead_key
and aead_nonce
.¶
The client uses these values to decrypt ct
using the Open function provided by
the AEAD. Decrypting might produce an error, as follows:¶
reponse, error = Open(aead_key, aead_nonce, "", ct)¶
A client interacts with the oblivious proxy resource by constructing an encapsulated request. This encapsulated request is included as the content of a POST request to the oblivious proxy resource. This request MUST only contain those fields necessary to carry the encapsulated request: a method of POST, a target URI of the oblivious proxy resource, a header field containing the content type (see (Section 7), and the encapsulated request as the request content. Clients MAY include fields that do not reveal information about the content of the request, such as Alt-Used [ALT-SVC], or information that it trusts the oblivious proxy resource to remove, such as fields that are listed in the Connection header field.¶
The oblivious proxy resource interacts with the oblivious request resource by constructing a request using the same restrictions as the client request, except that the target URI is the oblivious request resource. The content of this request is copied from the client. The oblivious proxy resource MUST NOT add information about the client to this request.¶
When a response is received from the oblivious request resource, the oblivious proxy resource forwards the response according to the rules of an HTTP proxy; see Section 7.6 of [HTTP].¶
An oblivious request resource, if it receives any response from the oblivious target resource, sends a single 200 response containing the encapsulated response. Like the request from the client, this response MUST only contain those fields necessary to carry the encapsulated response: a 200 status code, a header field indicating the content type, and the encapsulated response as the response content. As with requests, additional fields MAY be used to convey information that does not reveal information about the encapsulated response.¶
An oblivious request resource acts as a gateway for requests to the oblivious target resource (see Section 7.6 of [HTTP]). The one exception is that any information it might forward in a response MUST be encapsulated, unless it is responding to errors it detects before removing encapsulation of the request; see Section 6.2.¶
This encapsulation does not permit progressive processing of responses. Though the binary HTTP response format does support the inclusion of informational (1xx) status codes, the AEAD encapsulation cannot be removed until the entire message is received.¶
In particular, the Expect header field with 100-continue (see Section 10.1.1 of [HTTP]) cannot be used. Clients MUST NOT construct a request that includes a 100-continue expectation; the oblivious request resource MUST generate an error if a 100-continue expectation is received.¶
A server that receives an invalid message for any reason MUST generate an HTTP response with a 4xx status code.¶
Errors detected by the oblivious proxy resource and errors detected by the oblivious request resource before removing protection (including being unable to remove encapsulation for any reason) result in the status code being sent without protection in response to the POST request made to that resource.¶
Errors detected by the oblivious request resource after successfully removing encapsulation and errors detected by the oblivious target resource MUST be sent in an encapsulated response.¶
Media types are used to identify encapsulated requests and responses.¶
Evolution of the format of encapsulated requests and responses is supported through the definition of new formats that are identified by new media types.¶
The "message/ohttp-req" identifies an encapsulated binary HTTP request. This is a binary format that is defined in Section 5.1.¶
message¶
ohttp-req¶
N/A¶
None¶
only "8bit" or "binary" is permitted¶
N/A¶
this specification¶
N/A¶
N/A¶
see Authors' Addresses section¶
COMMON¶
N/A¶
see Authors' Addresses section¶
IESG¶
The "message/ohttp-res" identifies an encapsulated binary HTTP response. This is a binary format that is defined in Section 5.2.¶
message¶
ohttp-res¶
N/A¶
None¶
only "8bit" or "binary" is permitted¶
N/A¶
this specification¶
N/A¶
N/A¶
see Authors' Addresses section¶
COMMON¶
N/A¶
see Authors' Addresses section¶
IESG¶
In this design, a client wishes to make a request of a server that is authoritative for the oblivious target resource. The client wishes to make this request without linking that request with either:¶
In order to ensure this, the client selects a proxy (that serves the oblivious proxy resource) that it trusts will protect this information by forwarding the encapsulated request and response without passing the server (that serves the oblivious request resource).¶
In this section, a deployment where there are three entities is considered:¶
To achieve the stated privacy goals, the oblivious proxy resource cannot be operated by the same entity as the oblivious request resource. However, colocation of the oblivious request resource and oblivious target resource simplifies the interactions between those resources without affecting client privacy.¶
Clients MUST ensure that the key configuration they select for generating encapsulated requests is integrity protected and authenticated so that it can be attributed to the oblivious request resource; see Section 4.¶
Clients MUST NOT include identifying information in the request that is encapsulated.¶
Clients cannot carry connection-level state between requests as they only establish direct connections to the proxy responsible for the oblivious proxy resource. However, clients need to ensure that they construct requests without any information gained from previous requests. Otherwise, the server might be able to use that information to link requests. Cookies [COOKIES] are the most obvious feature that MUST NOT be used by clients. However, clients need to include all information learned from requests, which could include the identity of resources.¶
Clients MUST generate a new HPKE context for every request, using a good source of entropy ([RANDOM]) for generating keys. Key reuse not only risks requests being linked, reuse could expose request and response contents to the proxy.¶
The request the client sends to the oblivious proxy resource only requires minimal information; see Section 6. The request that carries the encapsulated request and is sent to the oblivious proxy resource MUST NOT include identifying information unless the client ensures that this information is removed by the proxy. A client MAY include information only for the oblivious proxy resource in header fields identified by the Connection header field if it trusts the proxy to remove these as required by Section 7.6.1 of [HTTP]. The client needs to trust that the proxy does not replicate the source addressing information in the request it forwards.¶
Clients rely on the oblivious proxy resource to forward encapsulated requests and responses. However, the proxy can only refuse to forward messages, it cannot inspect or modify the contents of encapsulated requests or responses.¶
The proxy that serves the oblivious proxy resource has a very simple function to perform. For each request it receives, it makes a request of the oblivious request resource that includes the same content. When it receives a response, it sends a response to the client that includes the content of the response from the oblivious request resource. When generating a request, the proxy MUST follow the forwarding rules in Section 7.6 of [HTTP].¶
A proxy can also generate responses, though it assumed to not be able to examine the content of a request (other than to observe the choice of key identifier, KDF, and AEAD), so it is also assumed that it cannot generate an encapsulated response.¶
A proxy MUST NOT add information about the client identity when forwarding requests. This includes the Via field, the Forwarded field [FORWARDED], and any similar information.¶
As there are privacy benefits from having a large rate of requests forwarded by the same proxy (see Section 8.2.2), servers that operate the oblivious request resource might need an arrangement with proxies. This arrangement might be necessary to prevent having the large volume of requests being classified as an attack by the server.¶
If a server does accept a large volume of requests from a proxy, it needs to trust that the proxy does not allow abusive levels of request volumes from clients. That is, if a server allows requests from the proxy to be exempt from rate limits, the server might want to ensure that the proxy applies similar rate limiting when receiving requests from clients.¶
Servers that enter into an agreement with a proxy that enables a higher request rate might choose to authenticate the proxy to enable the higher rate.¶
As the time at which encapsulated request or response messages are sent can reveal information to a network observer. Though messages exchanged between the oblivious proxy resource and the oblivious request resource might be sent in a single connection, traffic analysis could be used to match messages that are forwarded by the proxy.¶
A proxy could, as part of its function, add delays in order to increase the anonymity set into which each message is attributed. This could latency to the overall time clients take to receive a response, which might not what some clients want.¶
A proxy can use padding to reduce the effectiveness of traffic analysis.¶
A proxy that forwards large volumes of exchanges can provide better privacy by providing larger sets of messages that need to be matched.¶
A server that operates both oblivious request and oblivious target resources is responsible for removing request encapsulation, generating a response the encapsulated request, and encapsulating the response.¶
Servers should account for traffic analysis based on response size or generation time. Techniques such as padding or timing delays can help protect against such attacks; see Section 8.2.2.¶
If separate entities provide the oblivious request resource and oblivious target resource, these entities might need an arrangement similar to that between server and proxy for managing denial of service; see Section 8.2.1. It is also necessary to provide confidentiality protection for the unprotected requests and responses, plus protections for traffic analysis; see Section 8.2.2.¶
An oblivious request resource needs to have a plan for replacing keys. This might include regular replacement of keys, which can be assigned new key identifiers. If an oblivious request resource receives a request that contains a key identifier that it does not understand or that corresponds to a key that has been replaced, the server can respond with an HTTP 422 (Unprocessable Content) status code.¶
A server can also use a 422 status code if the server has a key that corresponds to the key identifier, but the encapsulated request cannot be successfully decrypted using the key.¶
Please update the "Media Types" registry at https://www.iana.org/assignments/media-types with the registration information in Section 7 for the media types "message/ohttp-req", "message/ohttp-res", and "application/ohttp-keys".¶
A single request and response exchange is shown here. Binary values (key configuration, secret keys, the content of messages, and intermediate values) are shown in hexadecimal. The request and response here are absolutely minimal; the purpose of this example is to show the cryptographic operations.¶
The oblivious request resource generates a key pair. In this example the server chooses DHKEM(X25519, HKDF-SHA256) and generates an X25519 key pair [X25519]. The X25519 secret key is:¶
15ce887006e079dcc7d67e73e5c13e31a55083f816eca9ebcf523b90ea2ab7b0¶
The oblivious request resource constructs a key configuration that includes the corresponding public key as follows:¶
0100200020f21c612398e4384c21b7f2a759133c6c2d1b9ce6d033613dfad2c7 3d4826214900080001000100010003¶
This key configuration is somehow obtained by the client. Then when a client
wishes to send an HTTP request of a GET request to https://example.com
, it constructs the following binary HTTP message:¶
00034745540568747470730b6578616d706c652e636f6d012f¶
The client then reads the oblivious request resource key configuration and selects a mutually supported KDF and AEAD. In this example, the client selects HKDF-SHA256 and AES-128-GCM. The client then generates an HPKE context that uses the server public key. This results in the following encapsulated key:¶
a7e0dffe93bc9ed807b51588f10669cd2f09ceb7f6a71153658275993eb88276¶
The corresponding private key is:¶
3b04f76e7ea1313484dd73c343adb94c23671f98cb66fc7ecc6127f38e1d4431¶
Applying the Seal operation from the HPKE context produces an encrypted message, allowing the client to construct the following encapsulated request:¶
0100010001a7e0dffe93bc9ed807b51588f10669cd2f09ceb7f6a71153658275 993eb88276b497362592977a4abc7857cf9892377af698ca0ee31fe72d21ca12 6ff2074eb9292ce7a83fc53158ff¶
The client then sends this to the oblivious proxy resource in a POST request, which might look like the following HTTP/1.1 request:¶
The oblivious proxy resource receives this request and forwards it to the oblivious request resource, which might look like:¶
The oblivous request resource receives this request, selects the key it generated previously using the key identifier from the message, and decrypts the message. As this request is directed to the same server, the oblivious request resource does not need to initiate an HTTP request to the oblivious target resource. The request can be served directly by the oblivious target resource, which generates a minimal response (consisting of just a 200 status code) as follows:¶
0140c8¶
The response is constructed by extracting a secret from the HPKE context:¶
861c67eefd91906068ee3208a9102274¶
The key derivation for the encapsulated response uses both the encapsulated KEM key from the request and a randomly selected nonce. This produces a salt of:¶
a7e0dffe93bc9ed807b51588f10669cd2f09ceb7f6a71153658275993eb88276 55645b6626d6e8b1163d739c6cd3d94a¶
The salt and secret are both passed to the Extract function of the selected KDF (HKDF-SHA256) to produce a pseudorandom key of:¶
ac29e0e1df48de4b349bb97db74da7f6732557a2fa6d12019ddadd8bf1e9cd99¶
The pseudorandom key is used with the Expand function of the KDF and an info field of "key" to produce a 16-byte key for the selected AEAD (AES-128-GCM):¶
0d81ce961eb6e61d510e18053418f316¶
With the same KDF and pseudorandom key, an info field of "nonce" is used to generate a 12-byte nonce:¶
ba2d3637b5f041cc15ef73b2¶
The AEAD Seal function is then used to encrypt the response, which is added to the randomized nonce value to produce the encapsulated response:¶
55645b6626d6e8b1163d739c6cd3d94ac7e0cfd48adf8057517241ee220d20bc e00a4a¶
The oblivious request resource then constructs a response:¶
The same response might then be generated by the oblivious proxy resource which might change as little as the Date header. The client is then able to use the HPKE context it created and the nonce from the encapsulated response to construct the AEAD key and nonce and decrypt the response.¶
This design is based on a design for oblivious DoH, described in [ODOH]. Eric Rescorla helped unify the structure of the key format.¶