JMAP Blob management extension

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 16 May 2022.¶

Copyright Notice

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶

Table of Contents

1. Introduction

Sometimes JMAP ([RFC8620]) interactions require creating a Blob and then referencing it. In the same way that IMAP Literals ([RFC7888]) were extended to reduce roundtrips for simple data, embedding simple small blobs into the JMAP method stream can reduce roundtrips.¶

Likewise, when fetching an object, it can be useful to also fetch the raw content of that object without a separate roundtrip.¶

Since raw blobs could contain arbitrary binary data, this document allows the use of the base64 coding specified in [RFC4648].¶

Where JMAP is being proxied through a system which applies additional access restrictions, it can be useful to know where a blob is referenced in order to decide whether to allow it to be downloaded, so this document defines a way to look up where a particular blobId is referenced.¶

2. Conventions Used In This Document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

3. Addition to the Capabilities Object

The capabilities object is returned as part of the JMAP Session object; see [RFC8620], Section 2.¶

This document defines an additional capability URI.¶

{
  "capabilities": {
    ...,
    "urn:ietf:params:jmap:blob": {}
  },
  "accounts": {
    "A13842": {
      ...
      "accountCapabilities": {
        "urn:ietf:params:jmap:blob": {
          "maxSizeBlobSet": 50000000,
          "maxCatenateItems": 100,
          "supportedTypeNames" : [
            "Mailbox",
            "Thread",
            "Email"
          ]
        }
      }
    }
  }
}

4. Blob Methods

A blob is a sequence of zero or more octets.¶

The JMAP base spec [RFC8620] defines the Blob/copy method, which is unchanged by this specfication, and is selected by the urn:ietf:params:jmap:core capability.¶

The following JMAP Methods are selected by the urn:ietf:params:jmap:blob capability.¶

Method Call:

[
  "Blob/set",
  {
    "accountId": "account1",
    "create": {
      "1": {
        "data:asBase64": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKA
                          AAAA1BMVEX/AAAZ4gk3AAAAAXRSTlN/gFy0ywAAAApJRE
                          FUeJxjYgAAAAYAAzY3fKgAAAAASUVORK5CYII=",
        "type": "image/png"
      },
    },
  },
  "R1"
]

Response:

[
  "Blob/set",
  {
    "accountId" : "account1",
    "created" : {
      "1": {
        "id" : "G4c6751edf9dd6903ff54b792e432fba781271beb",
        "type" : "image/png",
        "size" : 95
      },
    },
  },
  "R1"
]

Method Calls:

[
  [
    "Blob/set",
    {
      "create": {
        "b4": {
          "data:asText": "The quick brown fox jumped over the lazy dog."
        }
      }
    },
    "S4"
  ],
  [
    "Blob/set",
    {
      "create": {
        "cat": {
          "catenate": [
            {
              "data:asText": "How"
            },
            {
              "blobId": "#b4",
              "length": 7,
              "offset": 3
            },
            {
              "data:asText": "was t"
            },
            {
              "blobId": "#b4",
              "length": 1,
              "offset": 1
            },
            {
              "data:asBase64": "YXQ/"
            }
          ]
        }
      }
    },
    "CAT"
  ],
  [
    "Blob/get",
    {
      "properties": [
        "data:asText",
        "size"
      ],
      "ids": [
        "#cat"
      ]
    },
    "G4"
  ]
]

Responses:

[
  [
    "Blob/set",
    {
      "oldState": null,
      "created": {
        "b4": {
          "id": "Gc0854fb9fb03c41cce3802cb0d220529e6eef94e",
          "blobId": "Gc0854fb9fb03c41cce3802cb0d220529e6eef94e",
          "size": 45,
          "type": "application/octet-stream"
        }
      },
      "updated": null,
      "destroyed": null,
      "notCreated": null,
      "notUpdated": null,
      "notDestroyed": null,
      "accountId": "cassandane"
    },
    "S4"
  ],
  [
    "Blob/set",
    {
      "oldState": null,
      "created": {
        "cat": {
          "id": "Gcc60576f036321ae6e8037ffc56bdee589bd3e23",
          "blobId": "Gcc60576f036321ae6e8037ffc56bdee589bd3e23",
          "size": 19,
          "type": "application/octet-stream"
        }
      },
      "updated": null,
      "destroyed": null,
      "notCreated": null,
      "notUpdated": null,
      "notDestroyed": null,
      "accountId": "cassandane"
    },
    "CAT"
  ],
  [
    "Blob/get",
    {
      "list": [
        {
          "id": "Gcc60576f036321ae6e8037ffc56bdee589bd3e23",
          "data:asText": "How quick was that?",
          "size": 19
        }
      ],
      "notFound": [],
      "accountId": "cassandane"
    },
    "G4"
  ]
]

Method Call:

[
  "Blob/get",
  {
    "accountId" : "account1",
    "ids" : [
      "G6ec94756e3e046be78fcb33953b85b944e70673e",
      "not-a-blob"
    ],
    "properties" : [
      "data:asText",
      "data:asBase64",
      "size"
    ],
    "offset" : 4,
    "length" : 9
  },
  "R1"
]

Response:

[
  "Blob/get",
  {
    "accountId": "account1",
    "list": [
      {
        "id": "G6ec94756e3e046be78fcb33953b85b944e70673e",
        "data:asText": "quick bro",
        "data:asBase64": "cXVpY2sgYnJvCg==",
        "size": 46
      }
    ],
    "notFound": [
      "not-a-blob"
    ]
  },
  "R1"
]

[
  "Blob/lookup",
  {
    "typeNames": [
      "Mailbox",
      "Thread",
      "Email"
    ],
    "ids": [
      "Gd2f81008cf07d2425418f7f02a3ca63a8bc82003",
      "not-a-blob"
    ]
  },
  "R1"
]

[
  "Blob/lookup",
  {
    "list": [
      {
        "id": "Gd2f81008cf07d2425418f7f02a3ca63a8bc82003",
        "matchedIds": {
          "Mailbox": [
            "M54e97373",
            "Mcbe6b662"
          ],
          "Thread": [
            "T1530616e"
          ],
          "Email": [
            "E16e70a73eb4",
            "E84b0930cf16"
          ]
        }
      }
    ],
    "notFound": [
      "not-a-blob"
    ]
  },
  "R1"
]

5. Security considerations

JSON parsers are not all consistent in handling non-UTF-8 data. JMAP requires that all JSON data be UTF-8 encoded, so servers MUST only return a null value if data:asText is requested for a range of octets which is not valid UTF-8, and set isEncodingProblem: true.¶

Servers MUST apply any access controls, such that if the authenticated user would be unable to discover the blobId by making queries, then this fact can't be discovered via a Blob/lookup. For example, if an Email exists in a Mailbox which the authenticated user does not have access to see, then that emailId MUST not be returned in a lookup for a blob which is referenced by that email.¶

If a server might sometimes return all names empty rather than putting a blobId in the notFound response to a Blob/get, then the server SHOULD always return the same type of response, regardless of whether a blob exists but the user can't access it, or doesn't exist at all. This avoids leaking information about the existence of the blob.¶

The server MUST NOT trust that the data given to a Blob/set is a well formed instance of the specified media type, and if the server attempts to parse the given blob, only hardened parsers designed to deal with arbitrary untrusted data should be used. The server SHOULD NOT reject data on the grounds that it is not a valid specimen of the stated type.¶

Blob/set catenate can be used to recreate dangerous content on the far side of security scanners (anti-virus or exfiltration scanners for example) which may be watching the upload endpoint. Server implementations SHOULD provide a hook to allow security scanners to check the resulting blobId from a catenate in the same way that they do for the upload endpoint.¶

6. IANA considerations

7. Changes

EDITOR: please remove this section before publication.¶

The source of this document exists on github at: https://github.com/brong/draft-gondwana-jmap-blob/¶

draft-ietf-jmap-blob-06:¶

• removed asHex - we only need base64 and text¶
• added reference to where base64 is defined¶
• made 'destroy' not be allowed¶
• expanded JSON examples for readability¶
• removed 'expires' from examples¶

draft-ietf-jmap-blob-05:¶

• discovered I hadn't actually included typeNames and matchedIds anywhere except the updates section, oops!¶
• added a catenate example¶
• tightened up some text¶

draft-ieft-jmap-blob-04:¶

• added security considerations for scanning catenate results¶

draft-ieft-jmap-blob-03:¶

• added capabilities object¶
• renamed types to typeNames and matchedIds¶
• added details of how to handle non-UTF8 data and truncation in Blob/get¶
• added isTruncated and isEncodingProblem to Blob/get to tell the client if the request wasn't entirely satisfied.¶

draft-ieft-jmap-blob-02:¶

• fixed incorrect RFC number in reference and HTTP PUT -> POST, thanks Ken.¶
• added acknowledgements section¶
• removed all 'datatype' text and changed to 'data type' or 'type name' as appropriate (issue #1 proposal)¶
• expanded security considerations section and moved optional Blob/lookup empty case into Blob/lookup section¶

draft-ieft-jmap-blob-01:¶

• renamed 'datatypes' to 'types' to align with PushSubscription from RFC8620.¶
• added example for Blob/get¶
• specified offset and length precisely¶

draft-ieft-jmap-blob-00:¶

• initial adoption as an IETF document, otherwise identical to draft-gondwana-jmap-blob-02¶

draft-gondwana-jmap-blob-02¶

• renamed 'objects' to 'datatypes'¶
• specified Blob/lookup¶
• added IANA registry for datatypes¶

draft-gondwana-jmap-blob-01¶

• added an example¶

draft-gondwana-jmap-blob-00¶

• initial proposal¶

8. Acknowledgements

Joris Baum, Neil Jenkiuns, Alexey Melnikov, Ken Murchison, Robert Stepanek and the JMAP working group at the IETF.¶

Author's Address