Web Bundles

1.1. Terminology

Exchange (noun)

An HTTP request+response pair. This can either be a request from a client and the matching response from a server or the request in a PUSH_PROMISE and its matching response stream. Defined by Section 8 of [RFC7540].¶

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

1.2. Mode of specification

This specification defines how conformant bundle parsers work. It does not constrain how encoders produce a bundle: although there are some guidelines in Section 4, encoders MAY produce any sequence of bytes that a conformant parser would parse into the intended semantics.¶

This specification uses the conventions and terminology defined in the Infra Standard ([INFRA]).¶

2.1. Stream attributes and operations

• A sequence of available bytes. As the stream delivers bytes, these are appended to the available bytes.¶
• An EOF flag that's true if the available bytes include the entire stream.¶
• A current offset within the available bytes.¶
• A seek to offset N operation to set the current offset to N bytes past the beginning of the available bytes. A seek past the end of the available bytes blocks until N bytes are available. If the stream ends before enough bytes are received, either due to a network error or because the stream has a finite length, the seek fails.¶
• A read N bytes operation, which blocks until N bytes are available past the current offset, and then returns them and seeks forward by N bytes. If the stream ends before enough bytes are received, either due to a network error or because the stream has a finite length, the read operation returns an error instead.¶

2.2. Load a bundle's metadata

This takes the bundle's stream and returns either an error (where an error is a "format error" or a "version error"), an error with a fallback URL (which is also the primaryUrl when the bundle parses successfully), or a map ([INFRA]) of metadata containing at least keys named:¶

primaryUrl

The URL of the main resource in the bundle. If the client can't process the bundle for any reason, this is also the fallback URL, a reasonable URL to try to load instead.¶

requests

A map ([INFRA]) whose keys are URLs and whose values consist of either:¶

• A single ResponseMetadata value for a non-content-negotiated resource or¶

• A set of content-negotiated resources represented by¶

  • A Variants header field value ([I-D.ietf-httpbis-variants]) and¶
  • A map ([INFRA]) from each of the possible combinations of one available-value for each variant-axis to a ResponseMetadata structure. Load a response from a bundle can use the ResponseMetadata structures to find the matching response.¶

The map may include other items added by sections defined in the Web Bundle Section Name Registry.¶

This operation only waits for a prefix of the stream that, if the bundle is encoded with the "responses" section last, ends before the first response.¶

This operation's implementation is in Section 3.3.¶

2.3. Load a response from a bundle

This takes the stream of bytes representing the bundle, a request ([FETCH]), and the ResponseMetadata returned from Section 2.2 for the appropriate content-negotiated resource within the request's URL, and returns the response ([FETCH]) matching that request.¶

This operation can be completed without inspecting bytes other than those that make up the loaded response, although higher-level operations like proving that an exchange is correctly signed ([I-D.yasskin-http-origin-signed-responses]) may need to load other responses.¶

A client will generally want to load the response for a request that the client generated. For a URL with multiple variants, the client SHOULD use the algorithm in Section 4 of [I-D.ietf-httpbis-variants] to select the best variant.¶

This operation's implementation is in Section 3.4.¶

3.1. Top-level structure

This section is non-normative.¶

A bundle holds a series of named sections. The beginning of the bundle maps section names to the range of bytes holding that section. The most important section is the "index" (Section 3.3.1), which similarly maps serialized HTTP requests to the range of bytes holding that request's serialized response. Byte ranges are represented using an offset from some point in the bundle after the encoding of the range itself, to reduce the amount of work needed to use the shortest possible encoding of the range.¶

Future specifications can define new sections with extra data, and if necessary, these sections can be marked "critical" (Section 3.3.4) to prevent older parsers from using the rest of the bundle incorrectly.¶

The bundle is a CBOR item ([CBORbis]) with the following CDDL ([CDDL]) schema:¶

webbundle = [
  ; 🌐📦 in UTF-8.
  magic: h'F0 9F 8C 90 F0 9F 93 A6',
  version: bytes .size 4,
  primary-url: whatwg-url,
  section-lengths: bytes
    .cbor [* (section-name: tstr, length: uint) ],
  sections: [* any ],
  length: bytes .size 8,  ; Big-endian number of bytes in the bundle.
]

$section-name /=
  "index" / "manifest" / "signatures" / "critical" / "responses"

$section /= index / manifest / signatures / critical / responses

responses = [*response]

whatwg-url = tstr

3.2. Serving constraints

When served over HTTP, a response containing an application/webbundle payload MUST include at least the following response header fields, to reduce content sniffing vulnerabilities (Section 5.2):¶

• Content-Type: application/webbundle¶
• X-Content-Type-Options: nosniff¶

3.3. Load a bundle's metadata

A bundle holds a series of sections, which can be accessed randomly using the information in the section-lengths CBOR item, which holds a list of alternating section names and section lengths:¶

section-lengths = [* (section-name: tstr, length: uint) ],

To implement Section 2.2, the parser MUST run the following steps, taking the stream as input.¶

1. Seek to offset 0 in stream. Assert: this operation doesn't fail.¶
2. If reading 10 bytes from stream returns an error or doesn't return the bytes with hex encoding "86 48 F0 9F 8C 90 F0 9F 93 A6" (the CBOR encoding of the 6-item array initial byte and 8-byte bytestring initial byte, followed by 🌐📦 in UTF-8), return a "format error".¶
3. Let version be the result of reading 5 bytes from stream. If this is an error, return a "format error".¶
4. Let urlType and urlLength be the result of reading the type and argument of a CBOR item from stream (Section 3.5.3). If this is an error or urlType is not 3 (a CBOR text string), return a "format error".¶
5. Let fallbackUrlBytes be the result of reading urlLength bytes from stream. If this is an error, return a "format error".¶

6. Let fallbackUrl be the result of parsing ([URL]) the UTF-8 decoding of fallbackUrlBytes with no base URL. If either the UTF-8 decoding or parsing fails, return a "format error".¶

   Note: From this point forward, errors also include the fallback URL to help clients recover.¶

7. If version does not have the hex encoding "44 31 00 00 00" (the CBOR encoding of a 4-byte byte string holding an ASCII "1" followed by three 0 bytes), return a "version error" with fallbackUrl.¶

   Note: RFC EDITOR PLEASE DELETE THIS NOTE; Implementations of drafts of this specification MUST NOT use the version "1" in this byte string, and MUST instead define an implementation-specific string to identify which draft is implemented. This string SHOULD match the version used in the draft's MIME type (Section 6.1).¶

8. Let sectionLengthsLength be the result of getting the length of the CBOR bytestring header from stream (Section 3.5.2). If this is an error, return a "format error" with fallbackUrl.¶
9. If sectionLengthsLength is 8192 (8*1024) or greater, return a "format error" with fallbackUrl.¶
10. Let sectionLengthsBytes be the result of reading sectionLengthsLength bytes from stream. If sectionLengthsBytes is an error, return a "format error" with fallbackUrl.¶
11. Let sectionLengths be the result of parsing one CBOR item (Section 3.5) from sectionLengthsBytes, matching the section-lengths rule in the CDDL ([CDDL]) above. If sectionLengths is an error, return a "format error" with fallbackUrl.¶
12. Let (sectionsType, numSections) be the result of parsing the type and argument of a CBOR item from stream (Section 3.5.3).¶
13. If sectionsType is not 4 (a CBOR array) or numSections is not half of the length of sectionLengths, return a "format error" with fallbackUrl.¶

14. Let sectionsStart be the current offset within stream.¶

    For example, if sectionLengthsLength were 52 and sectionLengths contained 4 items (2 sections), sectionsStart would be 65 (10 initial bytes + a 2-byte bytestring header to describe a 52-byte bytestring + 52 bytes of section lengths + a 1-byte array header for the 2 sections).¶

15. Let knownSections be the subset of the Section 6.2 that this client has implemented.¶
16. Let ignoredSections be an empty set.¶
17. Let sectionOffsets be an empty map ([INFRA]) from section names to (offset, length) pairs. These offsets are relative to the start of stream.¶
18. Let currentOffset be sectionsStart.¶

19. For each ("name", length) pair of adjacent elements in sectionLengths:¶

    1. If "name"'s specification in knownSections says not to process other sections, add those sections' names to ignoredSections.¶

       Note: The ignoredSections enables sections that supercede other sections to be introduced in the future. Implementations that don't implement any such sections are free to omit the relevant steps.¶

    2. If sectionOffsets["name"] exists, return a "format error" with fallbackUrl. That is, duplicate sections are forbidden.¶
    3. Set sectionOffsets["name"] to (currentOffset, length).¶
    4. Set currentOffset to currentOffset + length.¶
20. If the "responses" section is not last in sectionLengths, return a "format error" with fallbackUrl. This allows a streaming parser to assume that it'll know the requests by the time their responses arrive.¶
21. Let metadata be a map ([INFRA]) initially containing the single key/value pair "primaryUrl"/fallbackUrl.¶

22. For each "name" --> (offset, length) triple in sectionOffsets:¶

    1. If "name" isn't in knownSections, continue to the next triple.¶
    2. If "name"'s Metadata field (Section 6.2) is "No", continue to the next triple.¶
    3. If "name" is in ignoredSections, continue to the next triple.¶
    4. Seek to offset offset in stream. If this fails, return a "format error" with fallbackUrl.¶
    5. Let sectionContents be the result of reading length bytes from stream. If sectionContents is an error, return a "format error" with fallbackUrl.¶
    6. Follow "name"'s specification from knownSections to process the section, passing sectionContents, stream, sectionOffsets, and metadata. If this returns an error, return a "format error" with fallbackUrl.¶
23. Assert: metadata has an entry with the key "primaryUrl".¶
24. If metadata doesn't have an entry with key "requests", return a "format error" with fallbackUrl.¶
25. Return metadata.¶

index = {* whatwg-url => [ variants-value, +location-in-responses ] }
variants-value = bstr
location-in-responses = (offset: uint, length: uint)

manifest = whatwg-url

signatures = [
  authorities: [*authority],
  vouched-subsets: [*{
    authority: index-in-authorities,
    sig: bstr,
    signed: bstr  ; Expected to hold a signed-subset item.
  }],
]
authority = augmented-certificate
index-in-authorities = uint

signed-subset = {
  validity-url: whatwg-url,
  auth-sha256: bstr,
  date: uint,
  expires: uint,
  subset-hashes: {+
    whatwg-url => [variants-value, +resource-integrity]
  },
  * tstr => any,
}
resource-integrity = (
  header-sha256: bstr,
  payload-integrity-header: tstr
)

critical = [*tstr]

 +------------+-----+----+----+----+----+----+----+----+----+----+
 | first byte | ... | 48 | 00 | 00 | 00 | 00 | 00 | BC | 61 | 4E |
 +------------+-----+----+----+----+----+----+----+----+----+----+
             /       \
       0xBC614E-10=12345668 omitted bytes

3.4. Load a response from a bundle

The result of Load a bundle's metadata maps each URL and Variant-Key ([I-D.ietf-httpbis-variants]) to a response, which consists of headers and a payload. The headers can be loaded from the bundle's stream before waiting for the payload, and similarly the payload can be streamed to downstream consumers.¶

response = [headers: bstr .cbor headers, payload: bstr]

To implement Section 2.3, the parser MUST run the following steps, taking the bundle's stream, a request ([FETCH]), and a responseMetadata returned by Section 2.2 .¶

1. Seek to offset responseMetadata.offset in stream. If this fails, return an error.¶
2. Read 1 byte from stream. If this is an error or isn't 0x82, return an error.¶
3. Let headerLength be the result of getting the length of a CBOR bytestring header from stream (Section 3.5.2). If headerLength is an error, return that error.¶
4. If headerLength is 524288 (512*1024) or greater, return an error.¶
5. Let headerCbor be the result of reading headerLength bytes from stream and parsing a CBOR item from them matching the headers CDDL rule. If either the read or parse returns an error, return that error.¶
6. Let (headers, pseudos) be the result of converting headerCbor to a header list and pseudoheaders using the algorithm in Section 3.6. If this returns an error, return that error.¶
7. If pseudos does not have a key named ':status' or its size isn't 1, return an error.¶
8. If pseudos[':status'] isn't exactly 3 ASCII decimal digits, return an error.¶
9. Let payloadLength be the result of getting the length of a CBOR bytestring header from stream (Section 3.5.2). If payloadLength is an error, return that error.¶

10. If payloadLength is greater than 0 and headers does not contain a Content-Type header, return an error.¶

    The client MUST interpret the following payload as this specified media type instead of trying to sniff a media type from the bytes of the payload, for example by appending an artificial X-Content-Type-Options: nosniff header field ([FETCH]) to headers.¶

11. If stream.currentOffset + payloadLength != responseMetadata.offset + responseMetadata.length, return an error.¶

12. Let body be a new body ([FETCH]) whose stream is a tee'd copy of stream starting at the current offset and ending after payloadLength bytes.¶

    TODO: Add the rest of the details of creating a ReadableStream object.¶

13. Let response be a new response ([FETCH]) whose:¶

    • Url list is request's url list,¶
    • status is pseudos[':status'],¶
    • header list is headers, and¶
    • body is body.¶
14. Return response.¶

3.5. Parsing CBOR items

Parsing a bundle involves parsing many CBOR items. All of these items need to be deterministically encoded.¶

3.6. Interpreting CBOR HTTP headers

Bundles represent HTTP requests and responses as a list of headers, matching the following CDDL ([CDDL]):¶

headers = {* bstr => bstr}

Pseudo-headers starting with a : provide the non-header information needed to create a request or response as appropriate¶

To convert a CBOR item item into a [FETCH] header list and pseudoheaders, parsers MUST do the following:¶

1. If item doesn't match the headers rule in the above CDDL, return an error.¶
2. Let headers be a new header list ([FETCH]).¶
3. Let pseudos be an empty map ([INFRA]).¶

4. For each pair (name, value) in item:¶

   1. If name contains any upper-case or non-ASCII characters, return an error. This matches the requirement in Section 8.1.2 of [RFC7540].¶

   2. If name starts with a ':':¶

      1. Assert: pseudos[name] does not exist, because CBOR maps cannot contain duplicate keys.¶
      2. Set pseudos[name] to value.¶
      3. Continue.¶
   3. If name or value doesn't satisfy the requirements for a header in [FETCH], return an error.¶

   4. Assert: headers does not contain ([FETCH]) name, because CBOR maps cannot contain duplicate keys and an earlier step rejected upper-case bytes.¶

      Note: This means that a response cannot set more than one cookie, because the Set-Cookie header ([RFC6265]) has to appear multiple times to set multiple cookies.¶

   5. Append (name, value) to headers.¶
5. Return (headers, pseudos).¶

5.1. Version skew

Bundles currently have no mechanism for ensuring that the signed exchanges they contain constitute a consistent version of those resources. Even if a website never has a security vulnerability when resources are fetched at a single time, an attacker might be able to combine a set of resources pulled from different versions of the website to build a vulnerable site. While the vulnerable site could have occurred by chance on a client's machine due to normal HTTP caching, bundling allows an attacker to guarantee that it happens. Future work in this specification might allow a bundle to constrain its resources to come from a consistent version.¶

5.2. Content sniffing

While modern browsers tend to trust the Content-Type header sent with a resource, especially when accompanied by X-Content-Type-Options: nosniff, plugins will sometimes search for executable content buried inside a resource and execute it in the context of the origin that served the resource, leading to XSS vulnerabilities. For example, some PDF reader plugins look for %PDF anywhere in the first 1kB and execute the code that follows it.¶

The application/webbundle format defined above includes URLs and request headers early in the format, which an attacker could use to cause these plugins to sniff a bad content type.¶

To avoid vulnerabilities, in addition to the response header requirements in Section 3.2, servers are advised to only serve an application/webbundle resource from a domain if it would also be safe for that domain to serve the bundle's content directly, and to follow at least one of the following strategies:¶

1. Only serve bundles from dedicated domains that don't have access to sensitive cookies or user storage.¶
2. Generate bundles "offline", that is, in response to a trusted author submitting content or existing signatures reaching a certain age, rather than in response to untrusted-reader queries.¶

3. Do all of:¶

   1. If the bundle's contained URLs (e.g. in the manifest and index) are derived from the request for the bundle, percent-encode ([URL]) any bytes that are greater than 0x7E or are not URL code points ([URL]) in these URLs. It is particularly important to make sure no unescaped nulls (0x00) or angle brackets (0x3C and 0x3E) appear.¶
   2. Similarly, if the request headers for any contained resource are based on the headers sent while requesting the bundle, only include request header field names and values that appear in a static allowlist. Keep the set of allowed request header fields smaller than 24 elements to prevent attackers from controlling a whole CBOR length byte.¶
   3. Restrict the number of items a request can direct the server to include in a bundle to less than 12, again to prevent attackers from controlling a whole CBOR length byte.¶
   4. Do not reflect request header fields into the set of response headers.¶

If the server serves responses that are written by a potential attacker but then escaped, the application/webbundle format allows the attacker to use the length of the response to control a few bytes before the start of the response. Any existing mechanisms that prevent polyglot documents probably keep working in the face of this new attack, but we don't have a guarantee of that.¶

To encourage servers to include the X-Content-Type-Options: nosniff header field, clients SHOULD reject bundles served without it.¶

6.1. Internet Media Type Registration

IANA is requested to register the MIME media type ([IANA.media-types]) for web bundles, application/webbundle, as follows:¶

• Type name: application¶
• Subtype name: webbundle¶

• Required parameters:¶

  • v: A string denoting the version of the file format. ([RFC5234] ABNF: version = 1*(DIGIT/%x61-7A)) The version defined in this specification is 1.¶

    Note: RFC EDITOR PLEASE DELETE THIS NOTE; Implementations of drafts of this specification MUST NOT use simple integers to describe their versions, and MUST instead define implementation-specific strings to identify which draft is implemented.¶

• Optional parameters: N/A¶
• Encoding considerations: binary¶
• Security considerations: See Section 5 of this document.¶
• Interoperability considerations: N/A¶
• Published specification: This document¶
• Applications that use this media type: None yet, but it is expected that web browsers will use this format.¶
• Fragment identifier considerations: N/A¶

• Additional information:¶

  • Deprecated alias names for this type: N/A¶
  • Magic number(s): 86 48 F0 9F 8C 90 F0 9F 93 A6¶
  • File extension(s): .wbn¶
  • Macintosh file type code(s): N/A¶
• Person & email address to contact for further information: See the Author's Address section of this specification.¶
• Intended usage: COMMON¶
• Restrictions on usage: N/A¶
• Author: See the Author's Address section of this specification.¶
• Change controller: The IESG iesg@ietf.org¶
• Provisional registration? Yes.¶

6.2. Web Bundle Section Name Registry

IANA is directed to create a new registry with the following attributes:¶

Name: Web Bundle Section Names¶

Review Process: Specification Required¶

Initial Assignments:¶

Requirements on new assignments:¶

Section Names MUST be encoded in UTF-8.¶

Assignments must specify whether the section is parsed during Load a bundle's metadata (Metadata=Yes) or not (Metadata=No).¶

The section's specification can use the bytes making up the section, the bundle's stream (Section 2.1), and the sectionOffsets map (Section 3.3), as input, and MUST say if an error is returned, and otherwise what items, if any, are added to the map that Section 3.3 returns. A section's specification MAY say that, if it is present, another section is not processed.¶