Extensible XDR Discriminated Union Primitive Type
draft-keiser-afs3-xdr-union-02

Abstract

AFS-3 relies upon XDR to carry Rx RPC call payloads. XDR discriminated unions are ill-suited to cases where the protocol needs to evolve without inventing new RPCs, i.e., unknown discriminant values cause the entire XDR payload to fail the decoding step. While this can be circumvented through the use of opaque payloads (and recursive XDR invocations), such solutions are inelegant and difficult to implement. This memo defines a new XDR primitive type, "ext-union", which is derived from the XDR discriminated union primitive type, but with two key variations: 1) each leg contains a length field, and 2) no default leg is supported.

Internet Draft Comments

Comments regarding this draft are solicited. Please include the AFS-3 protocol standardization mailing list (afs3-standardization@openafs.org) as a recipient of any comments.

Status of this Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on December 04, 2011.

Copyright Notice

Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

• 1. Introduction
• 1.1. Use Case
• 1.2. Abbreviations
• 2. Conventions
• 3. Extensible Discriminated Union
• 3.1. Extensible Union Type
• 3.2. RPC-L Changes
• 3.3. Encoding
• 3.4. Decoding
• 3.4.1. Error Handling
• 4. Acknowledgements
• 5. IANA Considerations
• 6. AFS Assign Numbers Registrar Considerations
• 7. Security Considerations
• 8. References
• 8.1. Normative References
• 8.2. Informative References
• Author's Address

1. Introduction

AFS-3 [AFS1] [AFS2] is a distributed file system that has its origins in the VICE project [CMU-ITC-84-020] [VICE1] at the Carnegie Mellon University Information Technology Center [CMU-ITC-83-025], a joint venture between CMU and IBM. VICE later became AFS when CMU moved development to a new commercial venture called Transarc Corporation, which later became IBM Pittsburgh Labs. AFS-3 is a suite of un-standardized network protocols based on a remote procedure call (RPC) suite known as Rx [AFS3-RX]. While de jure standards for AFS-3 fail to exist, the various AFS-3 implementations have agreed upon certain de facto standards, largely helped by the existence of an open source fork called OpenAFS that has served the role of reference implementation. In addition to using OpenAFS as a reference, IBM wrote and donated developer documentation that contains somewhat outdated specifications for the Rx protocol and all AFS-3 remote procedure calls, as well as a detailed description of the AFS-3 system architecture.

The Rx RPC protocol utilizes XDR [RFC4506] as its means of encoding RPC call and response payloads. XDR provides a discriminated union type. However, the semantics of the discriminated union base type do not lend themselves to evolution of the discriminant namespace: introduction of new discriminants--when there is no default leg--cause the remainder of the XDR octet stream to be un-parseable (due to the lack of a length field in the encoding) by older peers. This memo introduces a new XDR primitive type that is identical to the XDR discriminated union, except that: 1) each leg contains a length field, and 2) the default leg is disallowed.

1.1. Use Case

Given that this design doubles the overhead from 4 to 8 octets (relative to the XDR discriminated union primitive type), it is ill-suited for use with small implied legs. Within the AFS-3 protocol suite, the primary use case for the extensible union type is to wrap large data structures, rather than small primitive types.

1.2. Abbreviations

AFS -

Historically, AFS stood for the Andrew File System; AFS no longer stands for anything

RPC -

Remote Procedure Call

Rx -

The Remote Procedure Call mechanism utilized by AFS-3

XDR -

eXternal Data Representation

2. Conventions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

3. Extensible Discriminated Union

The extensible discriminated union will contain a length field in every leg so that decoding peers can compute the offset of the next object in the XDR octet stream, regardless of whether the discriminant is recognized. For small legs, this will result in significant encoding inefficiency, but it is necessary to permit the union to evolve over time (without peers failing to decode the entire XDR octet stream).

3.1. Extensible Union Type

The definition of the extensible discriminated union is derived from the XDR union defined in section 4.15 of the XDR specification [RFC4506]. Unlike XDR discriminated unions, the XDR types mapped to each arm of the union need not be defined a priori. Instead, the length of the arm is always included in the wire encoding along with the discriminant value, thus permitting the decoder to continue decoding past an unknown discriminant in an XDR octet stream.

How undefined discriminants are handled by the decoder is deliberately left unspecified by this document. Rather, this memo merely specifies which error conditions must be flagged to the caller (see Section 3.4.1). The error handling semantics--for both length mismatches and unknown discriminants--are left up to the definition of any type built upon the ext-union primitive type. While this lends significant flexibility to the design, it also permits XDR decoding to continue when the expected implied arm length doesn't match the length on the wire. It is RECOMMENDED that implementations fail to decode the entire XDR stream when such a length mismatch is encountered, as such a length mismatch is indicative of either: 1) a significant divergence in RPC-L definitions across the peers, or 2) an undetected bit error in the XDR octet stream.

Extensible discriminated unions are defined in RPC-L as follows:

    ext-union switch (discriminant-definition) {
    case discriminant-value-A:
       arm-declaration-A;
    case discriminant-value-B:
       arm-declaration-B;
    ...
    } identifier;
          

Because the discriminant namespace of an extensible union must be capable of evolving over time, it is not possible to support a default leg.

The extensible discriminated union is encoded on the wire as: a 4-octet discriminant, followed by a 4-octet arm length, and finally the variable-length implied arm. The arm length field SHALL count the length of the implied arm in octets, and only the implied arm. In other words, the 8 octets occupied by the discriminant and arm length fields SHALL NOT be counted as part of the arm length value.

  0   1   2   3
+---+---+---+---+---+---+---+---+---+---+---+---+
|  discriminant |   arm length  |  implied arm  |
+---+---+---+---+---+---+---+---+---+---+---+---+
|<---4 octets-->|<---4 octets-->|
          

Thus, in terms of existing XDR primitives, we can describe an extensible discriminated union as follows:

    struct ext_union {
        unsigned int discriminant;
        opaque implied_leg<>;
    };
          

It should be noted that this design makes it convenient to implement extensible discriminated unions on top of existing XDR primitive types.

3.2. RPC-L Changes

In order to implement the above, the XDR grammar, as specified in Section 6.3 of [RFC4506], will need to be modified in the following ways:

• "type-specifier" will require a new production rule mapping to "ext-union-type-spec", and
• an "ext-union-type-spec" production rule will need to be defined.

The "type-specifier" grammar will now include a new production rule for "ext-union-type-spec":

    type-specifier:
         [ "unsigned" ] "int"
       | [ "unsigned" ] "hyper"
       | "float"
       | "double"
       | "quadruple"
       | "bool"
       | enum-type-spec
       | struct-type-spec
       | union-type-spec
       | identifier
       | ext-union-type-spec
          

The new "ext-union-type-spec" production rule, and the production rule for its nonterminal symbol dependency "ext-union-body", are defined as follows:

    ext-union-type-spec:
       "ext-union" ext-union-body

    ext-union-body:
       "switch" "(" declaration ")" "{"
          case-spec
          case-spec *
       "}"
          

3.3. Encoding

It is RECOMMENDED that encoding of an AFS-3 extensible union proceed using the following algorithm:

1. encode an XDR unsigned 32-bit integer (see Section 4.2 of [RFC4506]) containing the discrimant,
2. XDR encode into temporary storage the implied leg (according to the type definition of the type specified for this discriminant value in the ext-union definition), and
3. encode the implied leg using the XDR variable-length opaque specification (see Section 4.10 of [RFC4506]).

3.4. Decoding

It is RECOMMENDED that decoding of an AFS-3 extensible union proceed using the following algorithm:

1. XDR decode the 32-bit unsigned integer containing the discriminant;
2. XDR decode the variable-length opaque blob into temporary storage;
3. If this is a known discriminant:
   1. Lookup the discriminant, and compare the expected length with the length of the previously-decoded XDR variable-length opaque;
   2. If the expected and actual lengths match:
      1. The implied leg's payload should be XDR decoded (according to the type definition of the type specified for this discriminant value in the ext-union definition);

   3. If the expected and actual lengths do not match:
      1. Mark the union as failed to decode due to a length mismatch;

4. However, if this is an unknown discriminant:
   1. Mark the union as failed to decode due to an unknown discriminant;

5. XDR decoding continues at the current offset plus the length of the previously-decoded XDR variable-length opaque.

3.4.1. Error Handling

While the specific decoding algorithm used is left up to the implementor, error handling MUST be implemented as described in this section.

Unknown Discriminant:

When a decoder encounters an unknown discriminant, it MUST mark the discriminant as unknown, and SHOULD proceed to decoding the next element in the XDR stream by seeking past the length and implied leg fields.

Unexpected Length for Discriminant:

When a decoder encounters a length field that doesn't agree with the length expected for this discriminant, it MUST mark the discriminant as failed to decode due to a length mismatch, and SHOULD fail to decode the rest of the XDR octet stream.

4. Acknowledgements

The author would like to thank Jeffrey Hutzelman for proposing standardization of a new XDR primitive type; and Andrew Deason, Simon Wilkinson, Derrick Brashear, and Matt Benjamin for helping to refine the design of this extensible union type. Finally, the author would like to thank the attendees of the 2011 Pittsburgh AFS Hackathon for their review and comments regarding the -01 draft.

5. IANA Considerations

This memo includes no request to IANA.

6. AFS Assign Numbers Registrar Considerations

This memo includes no request to the AFS Assigned Numbers Registrar.

7. Security Considerations

Users of this extensible type should understand that any Rx XDR payload is only as secure as the security class bound to the Rx connection in question. This document merely standardizes a primitive type; it is up to the authors of standards defining new types (upon the "ext-union" primitive type) to ensure that the contents of their types are only marshalled over sufficiently-secure security classes.

Decoders should take special care when encountering unexpected implied arm lengths. This could be indicative of serious errors, such as octet stream bit errors that were undetected by lower-layer checksums. At the very least, this error condition implies that the peers do not agree upon their ext-union type-to-discriminant mappings. It is RECOMMENDED that decoders treat this as a hard error and fail to decode the remainder of the XDR octet stream.

8. References

8.1. Normative References

8.2. Informative References

Author's Address