On storing CBOR encoded items on stable storage

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 8 March 2022.¶

Copyright Notice

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶

Table of Contents

1. Introduction

Since very early in computing, operating systems have sought ways to mark which files could be processed by which programs.¶

For instance, the Unix file(1) command, which has existed since 1973 [file], has been able to identify many file formats for decades based upon the contents of the file.¶

Many systems (Linux, MacOS, Windows) will select the correct application based upon the file contents, if the system can not determine it by other means. For instance, the classic MacOS maintained a resource fork that includes media type ("MIME type") information and therefore ideally never needs to know anything about the file.¶

But, many other systems do this by file extensions. Many common web servers derive the MIME-type information from file extensions.¶

While having a media type associated with the file is a better solution in general, when files become disconnected from their type information, such as when attempting to do forensics on a damaged system, then being able to identify a file type can become very important.¶

It is noted that in the media type registration, that a magic number is asked for, if available, as is a file extension.¶

A challenge for the file(1) program is often that it can be confused by the encoding vs the content. For instance, an Android "apk" used to transfer and store an application may be identified as a ZIP file. Additionaly, both OpenOffice or MSOffice files are ZIP files of XML files.¶

As CBOR becomes a more and more common encoding for a wide variety of artifacts, identifying them as just "CBOR" is probably not sufficient. This document provides a way to encode a magic number into the beginning of a CBOR format file. Two possible methods are presented: a CBOR Protocol author will specify one. (A CBOR Protocol is a specification which uses CBOR as its encoding)¶

Examples of CBOR Protocols currently under development include CoSWID [I-D.ietf-sacm-coswid], and EAT [I-D.ietf-rats-eat]. COSE itself [RFC8152] is considered infrastructure, however the encoding of public keys in CBOR as described in [I-D.ietf-cose-cbor-encoded-cert] would be an identified CBOR Protocol as well.¶

A major inspiration for this document is observing the mess in ASN.1 based systems where most files are PEM encoded, identified by the extension "pem", confusing public keys, private keys, certificate requests and S/MIME content.¶

While these proposals are invasive to how CBOR Protocols are stored in files, but in both cases, the proposed envelope does not require that the tag be transferred on the wire.¶

In addition to the on-disk identification aspects, there are some protocols which may benefit from having such a magic number on the wire if they are presently using a different (legacy) encoding scheme. The presence of the identifiable magic sequence signals that CBOR is being used vs a legacy scheme.¶

2. Requirements for a Magic Number

A magic number is ideally a unique fingerprint, present in the first 4 or 8 bytes of the file, which does not change when the contents change, and does not depend upon the length of the file.¶

Less ideal solutions have a pattern that needs to be matched, but in which some bytes need to be ignored. While the Unix file(1) command can be told to ignore bytes, this can lead to ambiguities.¶

3. Protocol

There are two methods presented. Which one is to be used is up to the CBOR Protocol author to determined. Both use CBOR Tags in a way that results in a deterministic first 8 to 12 bytes.¶

4. Advice to Protocol Developers

This document introduces a choice between a CBOR Sequence and a wrapped CBOR Tag. Which should a protocol designer use?¶

In this discussion, one assumes that there is an object stored in a file, perhaps specified by a system operator in a configuration file.¶

For example: a private key used in COSE operations, a public key/certificate in C509 or CBOR format, a recorded sensor reading stored for later transmission, or a COVID vaccination certificate that needs to be displayed in QRcode form.¶

Both the CBOR Tag Sequence and the wrapped tag can be trivially removed by an application before sending the CBOR content out on the wire.¶

The CBOR Tag Sequence is a little bit easier to remove as in most cases, CBOR parsers will return it as a unit, and then return the actual CBOR item, which could be anything at all, and could include CBOR tags that do need to be sent on wire.¶

On the other hand, having the CBOR Tag Sequence in the file requires that all programs that expect to examine that file are able to skip what appears to be an empty CBOR item. Programs which might not expect the CBOR Tag Sequence, but which would operate without a problem would include any program that expects to process CBOR Sequences from the file.¶

As an example of where there was a problem with previous security systems, "PEM" format certificate files grew to be able to contain multiple certificates by simple concatenation. The PKCS1 format could also contain a private key object followed by a one or more certificate objects: but only when in PEM format. Perversely, when in binary DER format (which like CBOR is self-limiting), concatenation of certificates was not compatible with most programs as they did not expect to read more than one item in the file.¶

The use of CBOR Tag Wrapped format is easier to retrofit to an existing format with existing and unchangeable on-disk format. This new sequence of tags are expected to be trivially ignored by an existing program when reading CBOR from disk, even if the program only supports decoding a single data item (and not a CBOR sequence). But, a naive program might also then transmit the additional tags across the network. Removing the CBOR Tag Wrapped format requires knowledge of the two tags involved. Other tags present might be needed.¶

For a representation matching a specific media-type that is carried in a CBOR byte string, the byte string head will already have to be removed for use as such a representation, so it should be easy to remove the enclosing tag heads as well. This is of particular interest with the pre-defined tags provided by Appendix A for media-types with CoAP Content-Format numbers.¶

Here are some considerations in the form of survey questions:¶

5. Security Considerations

This document provides a way to identify CBOR Protocol objects. Clearly identifying CBOR contents on disk may have a variety of impacts.¶

The most obvious is that it may allow malware to identify interesting objects on disk, and then corrupt them.¶

6. IANA Considerations

Section 6.1 documents the allocation that was done for a CBOR tag to be used in a CBOR sequence to identify the sequence (an example for using this tag is found in Appendix B). Section 6.2 allocates a CBOR tag for each actual or potential CoAP Content-Format number (examples are in Appendix A).¶

Appendix A. CBOR Tags for CoAP Content Formats

Often, there is a need to identify a media type (or content type, i.e., media type optionally used with parameters) that describes a byte string in a CBOR data item.¶

Section 5.10.3 of [RFC7252] defines the concept of a Content-Format, which is a short 16-bit unsigned integer that identifies a specific content type (media type plus optionally parameters), optionally together with a content encoding.¶

This specification allocates CBOR tag numbers 1668546560 (0x63740000) to 1668612095 (0x6374FFFF) for the tagging of representations of specific content formats. The tag content tagged with tag number NNNNNNNN (in above range) is a byte string that is to be interpreted as a representation of the content format NNNNNNNN-1668546560.¶

Appendix B. Example from Openswan

The Openswan IPsec project has a daemon ("pluto"), and two control programs ("addconn", and "whack"). They communicate via a Unix-domain socket, over which a C-structure containing pointers to strings is serialized using a bespoke mechanism. This is normally not a problem as the structure is compiled by the same compiler; but when there are upgrades it is possible for the daemon and the control programs to get out of sync by the bespoke serialization. As a result, there are extra compensations to deal with shutting the daemon down. During testing it is sometimes the case that upgrades are backed out.¶

In addition, when doing unit testing, the easiest way to load policy is to use the normal policy reading process, but that is not normally loaded in the daemon. Instead the IPC that is normally sent across the wire is compiled/serialized and placed in a file. The above magic number is included in the file, and also on the IPC in order to distinguish the "shutdown" command CBOR operation.¶

In order to reduce the problems due to serialization, the serialization is being changed to CBOR. Additionally, this change allows the IPC to be described by CDDL, and for any language that encode to CBOR can be used.¶

IANA has allocated the tag 1330664270, or 0x4f_50_53_4e for this purpose. As a result, each file and each IPC is prefixed a CBOR TAG Sequence.¶

In diagnostic notation:¶

55800(1330664270(h'424F52'))

Or in hex:¶

00000000  d9 d9 f8 da 4f 50 53 4e  43 42 4f 52         |....OPSNCBOR|

Appendix C. Changelog

Acknowledgements

The CBOR WG brainstormed this protocol on January 20, 2021 via a number of productive email exchanges on the mailing list.¶

Contributors

Authors' Addresses