Add LAYOUT_WCC to NFSv4.2's Flex File Layout Type

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 28 March 2025.¶

Copyright Notice

Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

In the Network File System version 4 (NFSv4) with a Parallel NFS (pNFS) Flexible File Layout (see Section 12 of [RFC8435]) server, there is no mechanism for the data servers to update the metadata servers for when the data portion of the file is modified. The metadata server needs this knowledge to correspondingly update the metadata portion of the file. If the client is using NFSv3 as the protocol with the data server, it can leverage weak cache consistency (WCC) to update the metadata server of the attribute changes. In this document, we introduce a new operation called LAYOUT_WCC to NFSv4.2 which allows the client to periodically report the attributes of the data files to the metadata server.¶

Using the process detailed in [RFC8178], the revisions in this document become an extension of NFSv4.2 [RFC7862]. They are built on top of the external data representation (XDR) [RFC4506] generated from [RFC7863].¶

2. Weak Cache Consistency (WCC)

A layout type for pNFS enables the metadata server to tell the client both the storage protocol and location of data to be used to communicate with the storage devices. The Flex Files Layout Type (in [RFC8435]) details how NFSv3 data servers can be accessed. The client is only allowed to perform NFSv3 READ (see Section 3.3.6 of [RFC1813]), WRITE (see Section 3.3.6 of [RFC1813]), and COMMIT (see Section 3.3.21 of [RFC1813]) operations on the file handles provided in the layout. I.e., the client is only allowed to use NFSv3 operations which directly act on the data portion of the data file.¶

Because there is no contol protocol (see [RFC8434]) possible with all data servers, NFSv3 is used as the control protocol. As such, the NFSv3 CREATE (see Section 3.3.8 of [RFC1813]), GETATTR (see Section 3.3.1 of [RFC1813]), and SETATTR (see Section 3.3.2 of [RFC1813]) are operations commonly used by the metadata server. I.e., the metadata server is only allowed to use NFSv3 operations which directly act on the metadata portion of the data file. GETATTR allows the metadata server to mainly retrieve the mtime (modify time), ctime (change time), and atime (access time). The metadata server can use this information to determine if the client modified the file whilst it held an iomode of LAYOUTIOMODE4_RW (see Section 3.3.20 of [RFC8881]). Then it can determine the time_modify (see Section 5.8.2.43 of [RFC8881]), time_metadata (see Section 5.8.2.42 of [RFC8881]), and time_access (see Section 5.8.2.37 of [RFC8881]) for the metadata file. I.e., the information to return to clients in a NFSv4.2 GETATTR response.¶

For example, the metadata server would issue a NFSv3 GETATTR to the data server. This query is most likely triggered in response to a NFSv4 GETATTR issued by a client to the metadata server. Not only are these NFSv3 GETATTRs to the data server individually expensive, the data server can become inundated by a storm of such requests. NFSv3 solved a similar issue by having the READ and WRITE operations employ a post-operation attribute to report the weak cache consistency (WCC) data (See Section 2.6 of [RFC1813]).¶

Each NFSv3 operation corresponds to one round trip between the client and server. So a WRITE followed by a GETATTR would require two round trips. In that scenario, the attribute information retrieved is considered to be strict server-client consistency. For NFSv4, the WRITE and GETATTR can be issued together inside a compound, which only requires one round trip between the client and server. And this is also considered to be a strict server-client consistency. In essence, the NFSv4 READ and WRITE operations drop the post-operation attributes, allowing the client to decide if it needs that information.¶

Whilst NFSv4 got rid of the requirement for WCC information to be supplied by the WRITE or READ operations, the introduction of pNFS re-introduces the same problem. The metadata server has to communicate with the data server in order to get at the data which could be provided by a WCC model.¶

With the flexible files layout type, the client can leverage the NFSv3 WCC to service the proxying of times (See Section 4 of [delstid]). But the granularity of this data is limited. With client side mirroring (See Section 8 of [RFC8435]), the client has to aggregate the N mirrored files in order to send one piece of information instead of N pieces of information. Also, the client is limited to sending that information only when it returns the delegation.¶

In this document, we present a new NFSv4.2 operation called LAYOUT_WCC, which allows the client to update the metadata server with information from the data server. The client is responsible for taking the NFSv3 WCC information (which is returned by the 3 operations it is allowed to use) and pass that back to the metadata server in the NFSv4.2 attributes. The metadata server MAY then avoid costly NFSv3 GETATTR calls to the data servers. As this is a weak model, the metatdate server MAY make such calls anyway in order to strengthen the model.¶

Can it go into LAYOUTRETURN?TH¶

3. Operation 77: LAYOUT_WCC - Layout Weak Cache Consistency

<CODE BEGINS>
/// struct LAYOUT_WCC4args {
///         stateid4        lowa_stateid;
///         layouttype4     lowa_type;
///         opaque          lowa_body<>;
/// };

<CODE ENDS>

<CODE BEGINS>
/// struct LAYOUT_WCC4res {
///         nfsstat4                lowr_status;
/// };

<CODE ENDS>

<CODE BEGINS>
/// struct ff_data_server_wcc4 {
///             deviceid4            ffdsw_deviceid;
///             stateid4             ffdsw_stateid;
///             nfs_fh4              ffdsw_fh_vers<>;
///             fattr4               ffdsw_attributes;
/// };
///
/// struct ff_mirror_wcc4 {
///             ff_data_server_wcc4  ffmw_data_servers<>;
/// };
///
/// struct ff_layout_wcc4 {
///             ff_mirror_wcc4       fflw_mirrors<>;
/// };


<CODE ENDS>

4. Extraction of XDR

This document contains the external data representation (XDR) [RFC4506] description of the new open flags for delegating the file to the client. The XDR description is embedded in this document in a way that makes it simple for the reader to extract into a ready-to-compile form. The reader can feed this document into the following shell script to produce the machine readable XDR description of the new flags:¶

<CODE BEGINS>
#!/bin/sh
grep '^ *///' $* | sed 's?^ */// ??' | sed 's?^ *///$??'


<CODE ENDS>

That is, if the above script is stored in a file called "extract.sh", and this document is in a file called "spec.txt", then the reader can do:¶

<CODE BEGINS>
sh extract.sh < spec.txt > layout_wcc.x


<CODE ENDS>

The effect of the script is to remove leading white space from each line, plus a sentinel sequence of "///". XDR descriptions with the sentinel sequence are embedded throughout the document.¶

Note that the XDR code contained in this document depends on types from the NFSv4.2 nfs4_prot.x file (generated from [RFC7863]). This includes both nfs types that end with a 4, such as offset4, length4, etc., as well as more generic types such as uint32_t and uint64_t.¶

While the XDR can be appended to that from [RFC7863], the various code snippets belong in their respective areas of the that XDR.¶

5. Security Considerations

There are no new security considerations beyond those in [RFC7862].¶

6. IANA Considerations

This section is to be removed before publishing as an RFC.¶

IANA should use the current document (RFC-TBD) as the reference for the new entries.¶

Authors' Addresses