A JSON Encoding for HTTP Field Values

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 4 March 2024.¶

Copyright Notice

Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

Defining syntax for new HTTP fields ([HTTP], Section 5) is non-trivial. Among the commonly encountered problems are:¶

• There is no common syntax for complex field values. Several well-known fields do use a similarly looking syntax, but it is hard to write generic parsing code that will both correctly handle valid field values but also reject invalid ones.¶
• The HTTP message format allows field lines to repeat, so field syntax needs to be designed in a way that these cases are either meaningful, or can be unambiguously detected and rejected.¶
• HTTP does not define a character encoding scheme ([RFC6365], Section 2), so fields are either stuck with US-ASCII ([RFC0020]), or need out-of-band information to decide what encoding scheme is used. Furthermore, APIs usually assume a default encoding scheme in order to map from octet sequences to strings (for instance, [XMLHttpRequest] uses the IDL type "ByteString", effectively resulting in the ISO-8859-1 character encoding scheme [ISO-8859-1] being used).¶

(See Section 16.3 of [HTTP] for a summary of considerations for new fields.)¶

This specification addresses the issues listed above by defining both a generic JSON-based ([RFC8259]) data model and a concrete wire format that can be used in definitions of new fields, where the goals were:¶

• to be compatible with field recombination when field lines occur multiple times in a single message (Section 5.3 of [HTTP]), and¶
• not to use any problematic characters in the field value (non-ASCII characters and certain whitespace characters).¶

2. Data Model and Format

In HTTP, field lines with the same field name can occur multiple times within a single message (Section 5.3 of [HTTP]). When this happens, recipients are allowed to combine the field line values using commas as delimiter, forming a combined "field value". This rule matches nicely JSON's array format (Section 5 of [RFC8259]). Thus, the basic data model used here is the JSON array.¶

Field definitions that need only a single value can restrict themselves to arrays of length 1, and are encouraged to define error handling in case more values are received (such as "first wins", "last wins", or "abort with fatal error message").¶

JSON arrays are mapped to field values by creating a sequence of serialized member elements, separated by commas and optionally whitespace. This is equivalent to using the full JSON array format, while leaving out the "begin-array" ('[') and "end-array" (']') delimiters.¶

The ABNF character names and classes below are used (copied from [RFC5234], Appendix B.1):¶

  CR               = %x0D    ; carriage return
  HTAB             = %x09    ; horizontal tab
  LF               = %x0A    ; line feed
  SP               = %x20    ; space
  VCHAR            = %x21-7E ; visible (printing) characters

Characters in JSON strings that are not allowed or discouraged in HTTP field values -- that is, not in the "VCHAR" definition -- need to be represented using JSON's "backslash" escaping mechanism ([RFC8259], Section 7).¶

The control characters CR, LF, and HTAB do not appear inside JSON strings, but can be used outside (line breaks, indentation etc.). These characters need to be either stripped or replaced by space characters (ABNF "SP").¶

Formally, using the HTTP specification's ABNF extensions defined in Section 5.6.1 of [HTTP]:¶

  json-field-value = #json-field-item
  json-field-item  = JSON-Text
                   ; see [RFC8259], Section 2,
                   ; post-processed so that only VCHAR characters
                   ; are used

5. Using this Format in Field Definitions

Specifications defining new HTTP fields need to take the considerations listed in Section 16.3 of [HTTP] into account. Many of these will already be accounted for by using the format defined in this specification.¶

Readers of HTTP-related specifications frequently expect an ABNF definition of the field value syntax. This is not really needed here, as the actual syntax is JSON text, as defined in Section 2 of [RFC8259].¶

A very simple way to use this JSON encoding thus is just to cite this specification -- specifically the "json-field-value" ABNF production defined in Section 2 -- and otherwise not to talk about the details of the field syntax at all.¶

An alternative approach is just to repeat the ABNF-related parts from Section 2.¶

This frees the specification from defining the concrete on-the-wire syntax. What's left is defining the field value in terms of a JSON array. An important aspect is the question of extensibility, e.g. how recipients ought to treat unknown field names. In general, a "must ignore" approach will allow protocols to evolve without versioning or even using entire new field names.¶

7. Interoperability Considerations

The "I-JSON Message Format" specification ([RFC7493]) addresses known JSON interoperability pain points. This specification borrows from the requirements made over there:¶

Appendix A. Comparison with Structured Fields

Appendix C. Implementations

This section is to be removed before publishing as an RFC.¶

See <https://github.com/reschke/json-fields> for a proof-of-concept (in development).¶

Appendix D. Change Log

This section is to be removed before publishing as an RFC.¶

Acknowledgements

Thanks go to the Hypertext Transfer Protocol Working Group participants.¶

Author's Address