Extended URLFETCH for binary and converted parts
draft-cridland-urlfetch-binary-03

Status of this Memo

By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on March 8, 2009.

Abstract

The URLFETCH command defined as part of URLAUTH provides a mechanism for third-parties to gain access to data held within messages in a user's private store, however, this data is sent verbatim, which is not suitable for a number of applications. This memo specifies a method for obtaining data in forms suitable for non-mail applications.

Table of Contents

1.  Introduction
2.  Conventions used in this document
3.  Extended URLFETCH
    3.1.  Command Parameters
    3.2.  Response Metadata
4.  Example Exchanges
5.  Formal Syntax
6.  IANA Considerations
7.  Security Considerations
8.  Acknowledgements
9.  References
    9.1.  Normative References
    9.2.  Informative References
§  Author's Address
§  Intellectual Property and Copyright Statements

1.  Introduction

Although [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.) provides a URLFETCH command which can be used to dereference a URL and return the body part data, it does so by returning the encoded form, without sufficient metadata to decode. This is suitable for use in mail applications such as [BURL] (Newman, C., “Message Submission BURL Extension,” May 2006.), where the encoded form is suitable, but not where access to the actual content is required, such as [STREAMING] (Cook, N., “Streaming Internet Messaging Attachments,” June 2008.).

This memo specifies a mechanism which returns additional metadata about the part, such as its [MEDIATYPE] (Freed, N. and N. Borenstein, “Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types,” November 1996.) type, as well as removing any content transfer encoding used on the body part.

2.  Conventions used in this document

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [KEYWORDS] (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.).

Protocol examples are line-wrapped for clarity. Protocol strings are prefixed with C: and S: for client and server respectively, and elided data is represented by [...]. Implementors should note these notations are for editorial clarity only.

3.  Extended URLFETCH

This extension is available in any IMAP server implementation which includes URLAUTH=BINARY within its capability string.

Such servers accept additional, per-URL, parameters to the URLFETCH command, and will provide, upon request, specific data for each URL dereferenced.

3.1.  Command Parameters

The URLFETCH command is extended by the provision of optional parameters. The extended URLFETCH command is distinct by enclosing each URL and associated parameters in a parenthesized list. The absence of any parameters, or if the URL is sent unenclosed, causes the command to behave precisely as specified in [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.).

Similarly, if the URL is invalid, the command will behave precisely as specified in [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.), and return a simple NIL.

Available parameters are:

BODYPARTSTRUCTURE

Provide a BODYPARTSTRUCTURE.

BODYPARTSTRUCTURE is defined in [CONVERT] (Melnikov, A. and P. Coates, “Internet Message Access Protocol - CONVERT Extension,” July 2008.), and provides metadata useful for processing applications, such as the type of the data.

BINARY

Provide the data without any Content-Transfer-Encoding.

In particular, this means that the data MAY contain NUL octets, and not be formed from textual lines. Data containing NUL octets MUST be transferred using the literal8 syntax defined in [BINARY] (Nerenberg, L., “IMAP4 Binary Content Extension,” April 2003.).

BODY

Provide the data as-is.

This will provide the same data as the unextended [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.) as a metadata item.

Metadata items MUST NOT appear more than once per URL requested, and clients MUST NOT request both BINARY and BODY.

3.2.  Response Metadata

In order to carry any requested metadata and provide additional information to the consumer, the URLFETCH response is similarly extended.

Following the URL itself, servers will include a series of parenthesized metadata elements. Defined metadata elements are as follows:

BODYPARTSTRUCTURE

The BODYPARTSTRUCTURE provides information about the data contained in the response, as it has been returned. It will reflect any conversions or decoding that have taken place. In particular, this will show an identity encoding if BINARY is also requested.

BINARY

The BINARY item provides the content, without any content transfer encoding applied. If this is not possible (for example, the content transfer encoding is unknown to the server), then this MAY contain NIL. Servers MUST understand all identity content transfer encodings defined in [MIME] (Freed, N. and N. Borenstein, “Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies,” November 1996.), as well as the transformation encodings "Base64" [BASE64] (Josefsson, S., “The Base16, Base32, and Base64 Data Encodings,” October 2006.) and "Quoted-Printable" [MIME] (Freed, N. and N. Borenstein, “Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies,” November 1996.).

BODY

The BODY item provides the content as found in the message, with any content transfer encoding still applied. Requesting only the BODY will provide equivalent functionality to the unextended [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.), however using the extended syntax described herein.

Note that unlike [CONVERT] (Melnikov, A. and P. Coates, “Internet Message Access Protocol - CONVERT Extension,” July 2008.), BODYPARTSTRUCTURE is not appended with the part specifier, as this is implicit in the URL.

4.  Example Exchanges

A client requests the data with no content transfer encoding.

C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
   section=1.2;urlauth=anonymous:internal:
   91354a473744909de610943775f92038" BINARY)
S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
   section=1.2;urlauth=anonymous:internal:
   91354a473744909de610943775f92038" (BINARY {28}
S: Si vis pacem, para bellum.
S:
S: )
S: A001 OK URLFETCH completed

Note that the data here does not contain a NUL octet, therefore a literal - not literal8 - syntax has been used.

A client again requests data with no content transfer encoding, but this time requests the body structure.

C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
   section=1.3;urlauth=anonymous:internal:
   ae354a473744909de610943775f92038" BINARY BODYPARTSTRUCTURE)
S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
   section=1.3;urlauth=anonymous:internal:
   ae354a473744909de610943775f92038" (BODYPARTSTRUCTURE
   ("IMAGE" "PNG" () NIL NIL "BINARY" 123)) (BINARY ~{123}
S: [123 octets of data, some of which is NUL])
S: A001 OK URLFETCH completed

A client requests only the body structure.

C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
   section=1.3;urlauth=anonymous:internal:
   ae354a473744909de610943775f92038" BODYPARTSTRUCTURE)
S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
   section=1.3;urlauth=anonymous:internal:
   ae354a473744909de610943775f92038" (BODYPARTSTRUCTURE
   ("IMAGE" "PNG" () NIL NIL "BASE64" 164))
S: A001 OK URLFETCH completed

A client requests the body structure, and the original content.

C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
   section=1.3;urlauth=anonymous:internal:
   ae354a473744909de610943775f92038" BODYPARTSTRUCTURE BODY)
S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
   section=1.3;urlauth=anonymous:internal:
   ae354a473744909de610943775f92038" (BODYPARTSTRUCTURE
   ("IMAGE" "PNG" () NIL NIL "BASE64" 164)) (BODY {164}
S: [164 octets of base64 encoded data])
S: A001 OK URLFETCH completed

Some parts cannot be decoded, so the server will provide the BODYPARTSTRUCTURE of the part as-is, and NIL for the binary content:

C: A001 URLFETCH ("imap://joe@example.com/INBOX/;uid=20/;
   section=1.4;urlauth=anonymous:internal:
   87ecbd02095b815e699503fc20d869c8" BODYPARTSTRUCTURE BINARY)
S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
   section=1.4;urlauth=anonymous:internal:
   87ecbd02095b815e699503fc20d869c8" (BODYPARTSTRUCTURE
   ("IMAGE" "PNG" () NIL NIL "X-BLURDYBLOOP" 123))
   (BINARY NIL)
S: A001 OK URLFETCH completed

If a part simply doesn't exist, however, or the URI is invalid for some other reason, then NIL is returned instead of metadata:

C: A001 URLFETCH ("imap://joe@example.com/INBOX/;uid=20/;
   section=200;urlauth=anonymous:internal:
   88066d37e2e5410e1a6486350a8836ee" BODYPARTSTRUCTURE BODY)
S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
   section=200;urlauth=anonymous:internal:
   88066d37e2e5410e1a6486350a8836ee" NIL
S: A001 OK URLFETCH completed

5.  Formal Syntax

This formal syntax uses ABNF as specified in [ABNF] (Crocker, D. and P. Overell, “Augmented BNF for Syntax Specifications: ABNF,” January 2008.), and includes productions defined in [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.), [BINARY] (Nerenberg, L., “IMAP4 Binary Content Extension,” April 2003.) and [IMAP] (Crispin, M., “INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1,” March 2003.).

capability       =/ "URLAUTH=BINARY"

   ; Command parameters; see Section 3.1

urlfetch         =  "URLFETCH" 1*(SP url-fetch-arg)

url-fetch-arg    =  url-fetch-simple / url-fetch-ext

url-fetch-simple =  url-full
   ; Unextended URLFETCH.

url-fetch-ext    =  "(" url-full *(SP url-fetch-param) ")"
   ; If no url-fetch-param present, as unextended.

url-fetch-param  =  "BODY" / "BINARY" / "BODYPARTSTRUCTURE" / atom

   ; Response; see Section 3.2

urlfetch-data    =  "*" SP "URLFETCH"
                    1*(SP (urldata-simple / urldata-ext /
                           urldata-error))

urldata-error    =  SP url-full SP nil

urldata-simple   =  SP url-full SP nstring
   ; If client issues url-fetch-simple, server MUST respond with
   ; urldata-simple.

urldata-ext      =  SP url-full url-metadata

url-metadata     =  1*(SP "(" url-metadata-el ")")

url-metadata-el  =  url-meta-bodystruct / url-meta-body /
                    url-meta-binary

url-bodystruct   =  "BODYPARTSTRUCTURE" SP body

url-binary       =  "BINARY" SP ( nstring / literal8 )
   ; If content contains a NUL octet, literal8 MUST be used.
   ; Otherwise, content SHOULD use nstring.
   ; On decoding error NIL should be used.

url-body         =  "BODY" SP nstring

6.  IANA Considerations

IMAP4 capabilities are registered by publishing a standards track or IESG approved experimental RFC. The registry is currently located at:

      http://www.iana.org/assignments/imap4-capabilities

This document defines the URLFETCH=BINARY IMAP capability. IANA is requested to add it to the registry accordingly.

7.  Security Considerations

Implementors are directed to the security considerations within [IMAP] (Crispin, M., “INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1,” March 2003.), [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.), and [BINARY] (Nerenberg, L., “IMAP4 Binary Content Extension,” April 2003.).

The ability of the holder of a URL to be able to fetch metadata about the content pointed to by the URL as well as the content itself allows a potential attacker to discover more about the content than was previously possible, including its original filename, and user-supplied description.

The additional value of this information to an attacker in marginal, and applies only to those URLs for which the attacker does not have direct access, such as those produced by [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.). Implementors are therefore directed to the security considerations present in [URLAUTH] (Crispin, M., “Internet Message Access Protocol (IMAP) - URLAUTH Extension,” May 2006.).

8.  Acknowledgements

Comments were received on the idea, and/or this draft, from Neil Cook, Philip Guenther, Alexey Melnikov, Ken Murchison and others. Whether in agreement or dissent, the comments have refined and otherwise influenced the document.

9.  References

9.1. Normative References

9.2. Informative References

Author's Address

Full Copyright Statement

Copyright © The IETF Trust (2008).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an “AS IS” basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.