Internationalization for the NFSv4 Protocols

2.1. Requirements Language Definition

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

2.2. Requirements Language Derivation

Although the key words "MUST", "SHOULD", and "MAY" retain their normal meanings, as described above, we need to explain how the statements involving these terms were arrived at:¶

• In the case of statements within Sections 10 and 13, these derive from the requirements of other internet specifications.¶
• In the case of statements within Sections 8, 6, and 9 derive from the author's view of the appropriate normative language to use and will, when this document is advanced, represent the working group's consensus on those same matters.¶
• However, in other cases, i.e. those in sections deriving from RFC7530 [RFC7530] (i.e. Sections 5, 7, 11, 12, 14, 15) this specification's descriptions were derived from existing implementation patterns. Although this pattern is atypical, it is needed to provide a description that satisfies the goal of [RFC2119], providing a normative description to enable future implementations to be compatible with existing ones. This requires that we explain later in this section how the normative terms used derive from the behavior of existing implementations, in those situations in which existing implementation behavior patterns can be determined.¶

Note that in introductory and explanatory sections of this document (i.e. Sections 1 through 4 these terms do not appear except to explain how they are used in this document. Also, they do not appear in Appendix B which provides non-normative implementation guidance.¶

With regard to the parts of this document deriving from RFC7530, we explain below how the normative terms used derive from the behavior of existing implementations, in those situations in which existing implementation behavior patterns can be determined.¶

• Behavior implemented by all existing clients or servers is described using "MUST", since new implementations need to follow existing ones to be assured of interoperability. While it is possible that different behavior might be workable, we have found no case where this seems reasonable.¶

  The revers holds for "MUST NOT": if a type of behavior poses interoperability problems, it has not been implemented by any existing clients or servers.¶

• Behavior implemented by most existing clients or servers, where that behavior is more desirable than any alternative, is described using "SHOULD", since new implementations need to follow that existing practice unless there are strong reasons to do otherwise.¶

  The converse holds for "SHOULD NOT".¶

• Behavior implemented by some, but not all, existing clients or servers is described using "MAY", indicating that new implementations have a choice as to whether they will behave in that way. Thus, new implementations will have the same flexibility that existing ones do.¶
• Behavior implemented by all existing clients or servers, so far as is known -- but where there remains some uncertainty as to details -- is described using "should". Such cases primarily concern details of error returns. New implementations should follow existing practice even though such situations generally do not affect interoperability.¶

There are also cases in which certain server behaviors, while not known to exist, cannot be reliably determined not to exist. In part, this is a consequence of the long period of time that has elapsed since the publication of the defining specifications, resulting in a situation in which those involved in the implementation work may no longer be involved in or be aware of working group activities.¶

In the case of possible server behavior that is neither known to exist nor known not to exist, we use "SHOULD NOT" and "MUST NOT" as follows, and similarly for "SHOULD" and "MUST".¶

• In some cases, the potential behavior is not known to exist but is of such a nature that, if it were in fact implemented, interoperability difficulties would be expected and reported, giving us cause to conclude that the potential behavior is not implemented. For such behavior, we use "MUST NOT". Similarly, we use "MUST" to apply to the contrary behavior.¶
• In other cases, potential behavior is not known to exist but the behavior, while undesirable, is not of such a nature that we are able to draw any conclusions about its potential existence. In such cases, we use "SHOULD NOT". Similarly, we use "SHOULD" to apply to the contrary behavior.¶

In the case of a "MAY", "SHOULD", or "SHOULD NOT" that applies to servers, clients need to be aware that there are servers that may or may not take the specified action, and they need to be prepared for either eventuality.¶

6.1. Handling of Canonical Equivalence of Strings

It is often desirable to treat two strings that are essentially the name, although normalized differently, as equivlent. Such equivalences can arise in multiple ways:¶

• In some cases, two Unicode values are assigned to a single glyph, because those two value represent different meanings of the same symbol. For example, OHM SIGN (U+2126) denotes the same symbol as GREEK CAPITAL LETTER OMEGA (U+03A9) and the two are considered canonically equivalent.¶

• There are a large number of situations in which a particular symbol can be represented as a single character or as a combination of a base character and a combining character adding a diacritic. For example, LATIN CAPITAL LETTER E ACUTE (U+00C9) can also represented by LATIN CAPITAL LETTER E (U+0045) followed by COMBINING ACTUE ACCENT (U+0301). These two strings are canonically equivalent.¶

  Generally, when such pairs exist, the form in which the diacritic is integrated into the sumbol is designated the NFC form while the other is the NFD form.¶

Whenever a set of at least teo canonically equivalent strings exists, one of these is one that is the NFC form and one is the NFD form. These are usually different although this is not always the case. Some examples:¶

1. OHM SIGN (U+2126) is canonically equivalent to GREEK CAPITAL LETTER OMEGA (U+03A9).¶

   In this case, the NFC and NFD forms are the the same and both are GREEK CAPITAL LETTER OMEGA (U+03A9).¶

2. The two strings LATIN CAPITAL LETTER E ACUTE (U+00C9) and LATIN CAPITAL LETTER E (U+0045) followed by COMBINING ACTUE ACCENT (U+0301) are canonically equivalent.¶

   In this case, the NFC form is LATIN CAPITAL LETTER E ACUTE (U+00C9) while the NFD form is LATIN CAPITAL LETTER E (U+0045) followed by COMBINING ACTUE ACCENT (U+0301).¶

3. The three strings ANGSTROM SIGN (U+212B), LATIN CAPITAL LETTER A WITH RING ABOVE (U+00C5), and LATIN CAPITAL LETTER A (U+0041) followed by COMBINING RING ABOVE (U+030A) are all canonically equivalent¶

   In this case, the NFC form is LATIN CAPITAL LETTER A WITH RING ABOVE (U+00C5) while the NFD form is LATIN CAPITAL LETTER A (U+0041) followed by COMBINING RING ABOVE (U+030A).¶

4. Sets of canonically eqivalent string can be arbitrarily large. For example, the twelve strings each consisting of one string from each of 1), 2), and 3) above are all canonically equivalent.¶

   In this case, the NFC form is of each of these twelve strings GREEK CAPITAL LETTER OMEGA (U+03A9) followed by LATIN CAPITAL LETTER E ACUTE (U+00C9) followed by LATIN CAPITAL LETTER A WITH RING ABOVE (U+00C5).¶

   In contrast, the NFD form of each of these twelve strings is GREEK CAPITAL LETTER OMEGA (U+03A9) followed by LATIN CAPITAL LETTER E (U+0045) followed by COMBINING ACTUE ACCENT (U+0301) followed by LATIN CAPITAL LETTER A (U+0041) followed by COMBINING RING ABOVE (U+030A).¶

While all of the above examples would be dealt with as stated above, regardless of the version of Unicode used by the server, the canonical equivalece relation is suject to change. This is because successive Unicode versions can add characters, creating instances of NFC form strings that did not exist previously.¶

In the context of NFSv4 servers, such equivalences can only be acted upon in the context of UTF8-aware file systems. In that context:¶

• Servers MAY map name strings other canonically equivalent strings, so that the naame of a file can be diffeent than the name specified by the user.¶

  Clients are expected to be tolerant of such mappings while many users are likely to consider canonically equivalent strings as being the same. Users who consider such strings as different would use UTF8-unaware file systems or those that did not modify user names.¶

• Servers MAY treat canonically equivalent srings as identical when searching for a given file without making any change in the names presented when the file is created.¶

  Clients are expected to be tolerant of such mappings while most users are likely to consider canonically equivalent strings as being the same. Users who consider these different would normally use UTF8-unaware file systems.¶

• While some other protcols deal with normalization issues by rejecting strings that are not in a particular normalization form, this option is not available to NFSv4 servers and NFsv4 clients are not required to abide by server-imposed normalization-form constraints¶

  Because the canonical equivalence relation can change, placing the burden of adapting to a particular normalization form and Unicode version would create a difficult-to-maintain file access API.¶

• Although clients can geneerally avoid any concern with the server's approach to normalization issues, there are, as described Section 6.3, some forms of client-side name caching for which the fact that the server treats two different strings as equivalent makes it desirable for the the client do so as well, or not use those forms of name caching.¶

  Because of the current inability of the client to determine the Unicode version used by the server, such forms of name caching are best avoided when using UTF8-aware file systems However Appendix B.4 discusses available possibilties for providing restrictions on such forms of name caching without eliminating them.¶

  For a discussion of how the client might be made aware of the specific canonical eqivalence relation used by the server, see Appendix A.4.¶

6.2. Handling of Case-insenitive Equivalence of Strings

In many envronments it is desirable to treat two strings as equivalent if they differ only as to case. This need arises when using operating environments in which file names are treated in a case-insensive manner. While determining whether two strings are equivalent except for case, can, in many environments, be a straightforward matter, there are, in internationalized enviromemts, situations in which user language preference or other similar considerations require the server implementer to make choices in this regard. See Appendix A.1 for a discussion of these cases.¶

In the context of NFSv4 servers, such equivalences can only be acted upon in the context of UTF8-aware file systems. In that context:¶

• Servers MAY map a name string to another string eqivalent except with regard to case, so that the naame of a file can be diffeent than the name requested by the user.¶

  When the OPTIONAL attributes case_insensitive and case_preserving are implemented, their values will both be false.¶

• Servers MAY treat name tring that only differ as to case as identical when searching for a given file without making any change in the name presented when the file is created.¶

  When the OPTIONAL attributes case_insensitive and case_preserving are implemented, their values will be true and false, respectively.¶

• Although clients can geneerally avoid any concern with the server's approach to case-handling issues, there are, as described Section 6.3, some forms of client-side name caching for which the fact that the server treats two different strings as equivalent make it desirable for the the client do so as well.¶

  Because of the current inability of the client to find out the details of the case equivalence relation use by the server, such forms of name caching are best avoided when using case-insensitive file systems. However Appendix B.4 discusses available possibilties for providing restrictions on such forms of name caching with eliinating them.¶

  For a discussion of how the client might be made aware of the case-eqivalence relation used by the server, see Appendix A.3.¶

6.3. String Equivalence and Client Name Caching

While most client functions are not affected by a seerver's implementation of various equivalence classes, there are a number of forms of name caching that require the client to be aware of string equivalence classes implemented by the server¶

• If the client implements ngeative name caching by caching the results of LOOKUP, OPEN, or ACCESS operations that find that the file does not exist, the server's treatment of two distinct strings as equivalent creates a potential problem.¶

  When negative name caching is implementd, there needs to be ways to eliminate records of the non-existence of particular files when they are no longer appropriate. This will occur when the files are found using LOOKUP, OPEN, or ACCESS or when names are added to the directory using OPEN, CREATE, LINK, or RENAME. When name equivalence relationship exist on the server, the client cannot act appropriately when files with previously non-existing name are found or created using distint names considered equivalent.¶

• If the client uses the results of earlier READDIR operations to enable later LOOKUP operations to be avoided, the efficiency of that caching is undercut when the client is unaware of the details of these equvalence relations.¶

  In such situations, the client's cached READDIR entry cannot be used, as it would on the server, to satisfy a LOOKUP for a dinstinct name equivalent to the first, requiring an over-the-wire operation that such caching is desired to avoid.¶

Because of these issues, when name equivalences are in effect, the above form of caching cannot work effectively and are best avoided.¶

8.1. The Attribute Fs_charset_cap in Published NFSv4.1 Specifications

We reproduce Section 14.4 of [RFC8881] below, with comments interspersed trying to make sense of what is there, in order to arrive at an appropriate replacement, to be presented in Section 8.2. In that connection, we need to understand better a few issues:¶

• The use of two bits while one is clearly adequate, given the subject matter actually mentioned.¶
• The mention of possible "capabilities" which could not possibly be realized.¶
• The use of the RFC2119 keyword "SHOULD" in contexts in which this term is clearly inappropriate.¶

   const FSCHARSET_CAP4_CONTAINS_NON_UTF8  = 0x1;
   const FSCHARSET_CAP4_ALLOWS_ONLY_UTF8   = 0x2;

   typedef uint32_t        fs_charset_cap4;

While it is made clear that two separate bits are to be provided, their names seem to indicate that they should be complements of one another. As a way of understanding why two bits were specified, it is helpful to consider a possible boolean attribute as a potential replacement. That attribute would clearly govern whether names that do not conform to the rules of UTF-8 are to be rejected, which was a "MUST" in [RFC3530]. Although conveying this information is clearly part of the motivation, stating so explicitly might have been judged by the authors as unnecessarily provocative, given the role of IESG in arriving at the internationalization approach specified in RFC3530.¶

• Because some operating environments and file systems do not enforce character set encodings,¶

It is clear that the ability of operating environments to enforce use of UTF-8 encoding is not an issue, since RFC3530 made this the responsibility of the server implementation. That mandate was never followed because implementers chose not to follow it, and not because they were unable to do so.¶

The apparently confused statement above is best understood if one notes that its essential job is to state that the "MUST" in RFC3530 referred to above is not reasonable. However, the authors might well have felt unable to say so explicitly, due to concerns about the potential IESG reaction.¶

• NFSv4.1 supports the fs_charset_cap attribute (Section 5.8.2.11) that indicates to the client a file system's UTF-8 capabilities.¶

The problem with the mention of (plural) capabilities is that the only capability mentioned which servers could implement is to accept strings which are not valid UTF-8. There are other potential capabilities having to do with the handling of canonically equivalent file nsmes, but since they were not mentioned, they will not be discussed further here.¶

• The attribute is an integer containing a pair of flags. The first flag is FSCHARSET_CAP4_CONTAINS_NON_UTF8, which, if set to one, tells the client that the file system contains non-UTF-8 characters,¶

As stated, this would mean that a server would have to keep track of a count of non-UTF-8-encoded names within the file system and change the attribute value as that count varied between zero and non-zero. Since it is most unlikely that any server would keep track of that or that any client would find it useful, we will assume that the capability to store such names is what is most likely intended.¶

• and the server will not convert non-UTF characters to UTF-8 if the client reads a symbolic link or directory,¶

There is no way for the server to convert non-UTF names to UTF-8 or anything else, since it has no knowledge of the name encoding to begin with. The alternative to treating names as UTF-8-encoded Unicode strings is to treat them as POSIX does, as uninterpreted strings of bytes. That makes it impossible to interpret strings that do not follow the rules of UTF-8 at all, making it impossible to convert the string to UTF-8.¶

• neither will operations with component names or pathnames in the arguments convert the strings to UTF-8.¶

As stated above, there is no way a server could ever do that.¶

• The second flag is FSCHARSET_CAP4_ALLOWS_ONLY_UTF8, which, if set to one, indicates that the server will accept (and generate) only UTF-8 characters on the file system.¶

That is clear and so it poses no problem for a revised treatment, unlike the other flag.¶

• If FSCHARSET_CAP4_ALLOWS_ONLY_UTF8 is set to one, FSCHARSET_CAP4_CONTAINS_NON_UTF8 MUST be set to zero.¶

There is no problem with this statement. However, it does, by implication, raise the issue of what values of FSCHARSET_CAP4_CONTAINS_NON_UTF8 may be set in the case in which FSCHARSET_CAP4_ALLOWS_ONLY_UTF8 is set to zero.¶

• FSCHARSET_CAP4_ALLOWS_ONLY_UTF8 SHOULD always be set to one.¶

According to xref target="RFC2119"/>, "SHOULD" means that "there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighing a different course". In this context, it is unclear what these "full implications" might be, given the introduction above. The clause, "because some operating environments and file systems do not enforce character set encodings", gives one no basis for treating this as other than an unproblematic behavioral variant, calling into question the use of "SHOULD".¶

Also, the statement in RFC2119 that these terms (i.e. those like "SHOULD") "only be used where it is actually required for interoperation or to limit behavior which has the potential for causing harm" raises the following issues:¶

• The whole purpose of this feature is to enable interoperation and there is no basis for the implication that one particular flag value is superior to another in allowing interoperation.¶
• There is no basis for assuming that accepting file names that are not UTF-8-encoded Unicode has any potential for causing harm.¶

Despite the statement in RFC2119, that "they [i.e. terms such as 'SHOULD'] must not be used to impose a particular method on implementors", it is hard to avoid the conclusion that this is in fact the motivation for the "SHOULD". The authors might not have had any such intention but it is posible they felt that the IESG might well have had such an intention and used "SHOULD" for this reason.¶

8.2. The Attribute Fs_charset_cap Going Forward

We provide a revised version of Section 14.4 of [RFC8881] below, taking into account the issues noted in Section 8.1. Given there was a working group consensus to adopt the confusing language discussed there, we must now adopt, by consensus, a clearer replacement that reflects the working group's intentions and is compatible with existing implementations. Given the passage of time and the changed context, it might not be possible to determine those intentions. In any case, we will have to be aware of how this attribute was implemented and used, particularly with regard to the first flag, whose meaning remains obscure.¶

The following treatment is proposed as a basis for discussion, with the understanding that it would need to be changed, if it could raise interoperability issues.¶

   const FSCHARSET_CAP4_CONTAINS_NON_UTF8  = 0x1;
   const FSCHARSET_CAP4_ALLOWS_ONLY_UTF8   = 0x2;

   typedef uint32_t        fs_charset_cap4;

• This attribute provides a simple way of determining whether a particular file system behaves as a UTF-8-only server and rejects file names which are not valid UTF8-encoded strings. When this attribute is supported and the value returned has the FSCHARSET_CAP4_ALLOWS_ONLY_UTF8 flag set, the error NFS4ERR_INVAL MUST be returned if any file name argument contains a string which is not a valid UTF8-encoded string.¶
• When this attribute is supported and the value returned has the FSCHARSET_CAP4_ALLOWS_ONLY_UTF8 flag clear, the error NFS4ERR_INVAL will not be returned based on adherence to the rules of UTF-8.¶
• With regard to the flag FSCHARSET_CAP4_CONTAINS_NON_UTF8, it has proved impossible to determine, from existing treatments of this attribute, any value that might be helpful here. As a result, we are forced to assume that this flag is always a complement of FSCHARSET_CAP4_ALLOWS_ONLY_UTF8 and that any result in which it is not so is to be ignored, with the appropriate handling being the same as would apply if the attribute were not supported.¶
• When this attribute is not supported, the client can perform a LOOKUP using a name not conforming to the rules of UTF-8 and use the error returned to determine whether non-UTF-8 names are accepted.¶

10.1. Effect of IDNA Changes

Overall, the effect of the shift to IDNA2008 is to limit the degree of understanding of the IDNA-based restrictions on domain names that were expected of NFSv4 in RFC7530 [RFC7530]. Despite this specification, the degree to which implementations actually implemented such restrictions is open to question. The consequences of this uncertainty will be discussed in detail in Section 10.2.¶

In analyzing how various cases are to be dealt with according to RFC7530, there a number of troubling uncertainties that arise in trying to interpret the existing specification:¶

• There are a number of cases in which "SHOULD" is used that are confusing. According to RFC2119 [RFC2119], "SHOULD" means that "there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course". To fully understand a particular "SHOULD", there needs to be enough context to determine whether particular reasons for ignoring the item are in fact valid, and sufficient guidance to understand the implication of ignoring the item. In the absence of such information, the relevant fact is that the peer needs to deal with the item being ignored, making the implications of a "SHOULD" hard to distinguish from those of "MAY".¶
• While the document states, "the general rules for handling all of these domain-related strings are similar and independent of the role of the sender or receiver as client or server", all of the following text is explicitly about the server's options, choices and responsibilities, leaving the client case unclear.¶
• In a number of places within the paragraph describing server approach #1, the word "can" is used as in the text "the server can use the ToUnicode function", leaving it unclear whether the server can choose to do anything else and if so what.¶

The following cases are those where RFC7530 requires use of IDNA handling and this requirement could, if implementations follow them, create potential compatibility issues, which need to be understood.¶

• The degree to which RFC3490 [RFC3490] requires that characters other than U+002E (full stop) be treated as label separators, including U+3002 (ideographic full stop), U+FF0E (fullwidth full stop), U+FF61 (halfwidth ideographic full stop).¶
• The degree to which RFC3490 [RFC3490] might require that server or client needs to validate a putative A-label or U-label or to rectify it if it is not valid.¶

10.2. Potential Compatibility Issues Related to IDNA Changes

There are a number of factors relating to the handling of domain names within NFSv4 implementations that are important in understanding why any compatibility issues might be less troubling than a comparison of the two IDNA approaches might suggest:¶

• Much of the potentially conflicting IDNA-related behavior required or recommended for the server by RFC7530 [RFC7530] appears to not be actually implemented, limiting the potential harmful effects of ceasing to mandate it.¶
• Even if such behavior were implemented by servers, no compatibility issue would arise unless clients actually relied on the server to implement it. Given that none of this behavior is made required, the chances of that occurring is quite small.¶

• The range of potential values for user and group attributes sent by clients are often quite small with implementations commonly restricting all such values to a single domain string. This is even though RFCs 7530 [RFC7530] and 5661 [RFC5661] are written without mention of such restrictions.¶

  Specification of users and groups in the "id@domain" format within NFSv4 was adopted to enable expansion of the spaces of users and groups beyond the 32-bit id spaces mandated in NFSv3 [RFC1813] and NFsv2 [RFC1094]. While one obstacle to expansion was eliminated, most implementations were unable to actually effect that expansion, principally because the physical file systems used assume that user and group identifiers fit in 32 bits each and the vnode interfaces used by server implementations make similar assumptions.¶

  Given these restrictions, the typical implementation pattern is for servers to accept only a single domain, specified as part of the server configuration, together with information necessary to effect the appropriate name-to-id mappings.¶

• For the other uses of domain names in NFSv4, to represent host names in location attributes, the values are generated by the server and will normally only include host names within DNS-registered domains.¶

Keeping the above in mind, we can see that interoperability issues, while they might exist, are unlikely to raise major challenges as looking to the following specific cases shows.¶

• When an internationalized domain name is used as part of a user or group, it would need to be configured as such, with the domain string known to both client and server.¶

  While it is theoretically possible that a client might work with an invalid domain string and rely on the server to correct it to an IDNA-acceptable one, such a scenario has to be considered extremely unlikely, since it would depend on multiple servers implementing the same correction, especially since there is no evidence of such corrections ever having been implemented by NFSv4 servers.¶

• When an internationalized domain in a location string is meant to specify a registered domain, similar considerations apply.¶

  While it is theoretically possible that a client might work with an invalid domain string and rely on the server to correct it to an appropriate registered one, such a scenario has to be considered extremely unlikely, since it would depend on multiple servers implementing the same correction, especially since there is no evidence of such corrections ever having been implemented by NFSv4 servers.¶

• When an internationalized domain in a location string is meant to specify a non-registered domain, any such server-applied corrections would be useless.¶

  In this situation, any potential interoperability issue would arise from rejecting the name, which has to be considered as what should have been done in the first place.¶

A.1. Important Issues for Case-insensitive Handling of File Names

In this section, we discuss many of the interesting and/or troublesome issues that the need for case-insensitive handling gives rise to in fully internationalized environments. Many of these are also discussed in [UNICODE-CASEM]. However, our treatment of these issues, while not inconsistent with that in [UNICODE-CASEM], differs significantly for a number of reasons:¶

• Our primary focus is on case-insensitive string comparison rather than with case mapping per se. While such comparison is natural for the client and allowed for servers, its greater flexibility makes it important to understand its capabilities in dealing with potentially troublesome issues in providing case-insensitive file name handling.¶
• Because a case mapping model forces the specification of a single case mapping result when there are multiple potentially valid results, there are inevitably cases in which the result chosen is inappropriate for some users. These are cases in which F-type and S-type mappings are present and in which C-type and T-type mappings conflict. Normally, an appropriate choice is selected by use of the locale, but in a file system environment, valid locale information might not be present. As a result, case-insensitive string comparison, which does not force such case mapping choices, will be more desirable since it allows construction of sets of equivalent strings based on muliple mappings which is not possible when case mappingis the goal.¶

The examples below present common situations that go beyond the simple invertible case mappings of Latin characters and the straightforward adaptation of that model to Greek and Cyrillic. In EX4 and EX5 we have case-based sets of equivalent strings including multi-character strings not derived from canonical equivalences while for EX7 and EX8 all multi-character strings are derived from canonical equivalences. In addition, EX1, EX2, EX3 and EX6 discuss other situations in which a set of equivalent strings has more than two elements.¶

EX1:

Certain digraph characters such LATIN SMALL LETTER DZ (U+01F3) have additional case variants to consider such as the titlecase character LATIN CAPTAL LETTER D WITH SMALL LETTER Z (U+01F2) in addition to the uppercase LATIN CAPITAL LETTER DZ (U+01F1). While the titlecased variant would not appear in names in case-insensitive non-case-preserving file systems, case-insensitive string comparison has no problem in treating these three characters as within same se of equivalent characters.¶

This set of equivalent strings can be derived using only C-type mappings. The possibility of mapping these characters to the two-character sequences they represent is not a troublesome issue since that would be derived from a compatibility equivalence, rather than a canonical equivalence, and there is no F-type mapping making it an option.¶

EX2:

To deal with the case of the OHM SIGN (U+2126) which is essentially identical to the GREEK CAPITAL LETTER OMEGA (U+03A9), one can construct an set of equivalent characters consisting of OHM SIGN (U+2126), GREEK CAPITAL LETTER OMEGA (U+03A9), and GREEK SMALL LETTER OMEGA (U+03C9).¶

This set of equivalent strings can be derived using only C-type mappings. Both OHM SIGN (U+2126), and GREEK CAPITAL LETTER OMEGA (U+03A9) lowercase to GREEK LETTER OMEGA (U+03C9), while that character only uppercases to GREEK CAPITAL LETTER OMEGA (U+03A9).¶

EX3:

To deal with the case of the ANGSTROM SIGN (U+212B) which is essentially identical to LATIN CAPITAL LETTER A WITH RING ABOVE (U+00C5), one can construct a set of equivalent strings consisting of ANGSTROM SIGN (U+212B), LATIN CAPITAL LETTER A WITH RING ABOVE (U+00C5), LATIN SMALL LETTER A WITH RING ABOVE (U+00E5), together with the two-character sequences involving LATIN CAPITAL LETTER A (U+0041) or LATIN SMALL LETTER A (U+0061) followed by COMBINING RING ABOVE (U+030A).¶

This set of equivalent strings can be derived using only C-type mappings together with the ability to map characters to canonically equivalent strings. Both ANGSTROM SIGN (U+212B), and LATIN CAPITAL LETTER A WITH RING ABOVE (U+00C5) lowercase to LATIN SMALL LETTER A WITH RING ABOVE (U+00E5), while that character only uppercases to CAPITAL LETTER A WITH RING ABOVE (U+00C5).¶

EX4:

In some cases, case mapping of a single character will result in a multi-character string. For example, the German character LATIN SMALL LETTER SHARP S (U+00DF) would be uppercased to "SS", i.e. two copies of LATIN CAPITAL LETTER S (U+0053). On the other hand, in some situations, it would be uppercased to the character LATIN CAPITAL LETTER SHARP S (U+1E9E), using an S-type mapping, referred to as an instance of "Tailored Casing". Unfortunately, in the context of a file system, there is unlikely to be available information that provides guidance about which of these case mappings should be chosen. However, the use of case-insensitive mappings with larger equivalence classes often provides handling that is acceptable to a wider variety of users. In this case, if noth mappings were usedd together to create a set of equivalent strings, German-speakers would get the mapping they expect while those unfamiliar with these characters only see them when they access a file whose name contains such characters.¶

It appears that if the construction of case-based equivalence classes were generalized to include multi-character sequences, then all of LATIN SMALL LETTER SHARP S (U+00DF), LATIN CAPITAL LETTER SHARP S (U+1E9E), "ss", "sS", "Ss", and "SS" would belong to the same equivalence class and could be handled by the general algorithm described in Appendix B.1, rather than by code specifically written to deal with this particular issue, which might hard to maintain.¶

EX5:

Other ligatures, such as LATIN SMALL LIGATURE FFL (U+FB04), could be handled similarly by this algorithm, if there were felt to be a need to do so. However, because the decomposition of this character into the string consisting of the three letters LATIN SMALL LETTER F (U+0066), LATIN SMALL LETTER F (U+0066), LATIN SMALL LETTER L (U+006C), is a compatibility equivalence, and the F-type mapping of this ligature to the three constituent characters is to be treated as optional, implementations can choose either to treat this character as having no uppercase equivalent or treat it as part of larger set of equivalent strings including "ffl", "ffL", "fFl", etc.).¶

EX6:

The character COMBINING GREEK YPOGEGRAMMENI (U+0345), also known as "iota-subscript" requires special handling when uppercasing and lowercasing. While the description of the appropriate handling for this character, in the case mapping section, is focused on multi- character sequences representing diphthongs, case-insensitive comparisons can be performed without consideration of multi-character sequences. This can be done by assigning COMBINING GREEK YPOGEGRAMMENI (U+0345), GREEK SMALL LETTER IOTA (U+03B9), and GREEK CAPITAL LETTER IOTA (U+0399) to the same equivalence class, even though the first of these is a combining character and the others are not.¶

EX7:

In some cases, context-dependent case mapping is required. For example, GREEK CAPITAL LETTER SIGMA (U+03A3) lowercases to GREEK SMALL LETTER SIGMA (U+03C3) if it is followed by another letter and to GREEK SMALL LETTER FINAL SIGMA (U+03C2) if it is not.¶

Despite this, case-insensitive comparisons can be implemented, by considering all of these characters as part of the same equivalence class, without any context-dependence, and this set of equivalent strings can be be derived using only C-type mappings.¶

EX8:

In most languages written using Latin characters, the uppercase and lowercase varieties of the letter "I" map to one nother. In a number of Turkic languages, there are two distinct characters derived from "I" which differ only with regard to the presence or absence of a dot so that there are both capital and small i's with each having dotted and dotless variants. Within such languages, the dotted and dotless I's represent different vowel sounds and are treated as separate characters with respect to case mapping. The uppercase of LATIN SMALL LETTER I (U+0069) is LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130), rather than LATIN CAPITAL LETTER I (U+0049). Similarly the lowercase of LATIN CAPITAL LETTER I (U+0049) is LATIN SMALL LETTER DOTLESS I (U+0131) rather than LATIN SMALL LETTER I (U+0069).¶

When doing case mapping, the server must choose to uppercase LATIN SMALL LETTER I (U+0069) to either LATIN CAPITAL LETTER I (U+0049), based on a C-type mapping to LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130), based on a T-type mapping. The former is acceptable to most people but confusing to speakers of the Turkic languages in question since the case mapping changes the character to represent a different vowel sound. On the other hand, the latter mapping seemingly inexplicably results in a character many users have never seen before. Normally such choices are dealt with based on a locale but, in a file system environment, no locale information is likely to be available.¶

In the context of case-insensitive string comparison, it is possible to create a larger set of equivalent strings, including all of the letters LATIN SMALL LETTER I (U+0069), LATIN CAPITAL LETTER I (U+0049), LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130), LATIN SMALL LETTER DOTLESS I (U+0131) together with the two-character string consisting of LATIN CAPITAL LETTER I (U+0049) followed by COMBINING DOT ABOVE (U+0307).¶

A.2. Defining Case-Insensitive Processing of File Names

When a server implements case-insensitive file name handling, it is desirable that clients do so as well. For example, if a client possessing the cached contents of a directory, notes that the file "a" does not exist, it cannot immediately act on that presumed non-existence, without checking for the potential existence of "A" as well. As a result, clients, in order to do certain form of name caching, might need to be able to provide case-insensitive name comparisons, irrespective of whether the server handling is case-preserving or not.¶

Because case-insensitive name comparisons are not always as straightforward as the above example suggests, the client, if it is to emulate the server's name handling, would need information about how certain cases are to be dealt with. In cases in which that information is unavailable, the client needs to avoid making assumptions about the server's handling, since it will be unaware of the Unicode version implemented by the server, or many of the details of specific issues that might need to be addressed differently by different server file systems in implementing case-insensitive name handling.¶

Many of the problematic issues with regard to the case-insensitive handling of names are discussed in Section 5.18 of the Unicode Standard [UNICODE-CASEM] which deals with case mapping. While we need to address all of these issues as well, our approach will not be exactly the same.¶

• Since the client would only need to to be doing case-insensitive comparisons, issues that apply only to uppercasing or lowercasing do not have the same significance.¶
• Many clients will have to operate correctly even in the absence of detailed information about the specifics of server-side case-mapping or the version of Unicode implemented by the server.¶
• Clients will have to accommodate server behaviors not anticipated by the Unicode Specification since it might be that neither the server nor the client would have any relavant locale knowledge when file names are processed.¶

Another source of information about case-folding, and indirectly about case-insensitive comparisons, is the case-folding text file which is part of the Unicode Standard [UNICODE-CASEF]. This file contains, for each Unicode character that can be uppercased or lowercased, a single character, or, in some cases a string of characters of the other case. For characters in capital case, the lowercase counterpart is given. Each of the mappings is characterized as of one of four types:¶

• Common case folding, denoted by a status field of "C". These are used for mapping where a single character can be mapped to a single character of another case. These are always valid with one potential exception being the mappings of LATIN CAPITAL LETTER I to LATIN SMALL LETTER I and vice versa, which might be superseded by the T-type mappings associated with some Turkic languages when written using Latin letters.¶
• Full case folding, denoted by a status field of "F". These are used for mappings in which single character is mapped to a multi-character string of a different case.¶
• Special case folding, denoted by a status field of "S". These provide additional single-character-to-single-character which might be used when there is also an F-type mapping of the same character. In the case of case folding, this is an alternative to the corresponding F-type, although, for the purposes of case-insensitive string comparison, it is possible for both to be considered valid at the same time¶
• Special case foldings for Turkic languages, denoted by a status field of "T". These consist of the invertible case mappings between LATIN SMALL LETTER I (U+0069) and LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130) and between LATIN CAPITAL LETTER I (U+0049) and LATIN SMALL LETTER DOTLESS I (U+0131). The relationship between these mappings and the C-type mappings for LETTER I is discussed below in item EX8.¶

While the case mapping section does discuss case-insensitive string comparisons, and describes a procedure for constructing equivalence classes of Unicode characters, the description does not deal clearly with the effect of F-type mappings. There are a number of problems with dealing with F-type mappings for case folding and basing case-insensitive string comparisons on those mappings, particularly in situations, such as file systems, in which extensive processing of strings is unlikely to be practical.¶

• Mappings from single characters to multi-character strings, are, for case-folding purposes, not invertible. However, case-insensitive name comparison, by its nature, requires invertible mappings, in which a multi-character string is mapped to a single character of a different case. This is not compatible with any existing simple case-mapping model.¶
• Scanning of names for multi-character sequences might well be too complicated for effective implemenntation within a file system, especially since such sequences might overlap in complicated ways.¶
• Case foldings which map single characters to multi-character sequences (see item EX4 below for an important example), would give rise, because of the invertibility of case mappings when used to determine case-insensitive string equivalence for very large sets of strings. For example, a string of eight copies of the letter S would give rise to an set of 256 equivalent strings plus over two thousand others when the German SHARP S characters discussed in item EX4 are included.¶

Despite these potential difficulties, case mappings involving multi-character sequences can be reversed when used as a basis for case-insensitive string comparisons and incorporated into a set of equivalence classes on name strings, as described below.¶

• Case-insensitive servers MAY do either case-mapping to a chosen case (the non-cas-preserving case), or case-insensitive string comparisons when providing a case-preserving implementation. In either case, the server MAY include F-type mappings, which map a single character to a multi-character string. However, only the case in which it is doing case-insensitive string comparison will it use the inverse of F-type mappings, in which a multi-character string is mapped to a single character of a different case¶

  In these cases, the server can choose to use either a C-type mapping or an F-type mapping, or both, when both exist. Similarly the server may choose to implement the C-type mappings of LATIN CAPITAL LETTER I to LATIN SMALL LETTER I and vice versa, the corresponding T-type mappings or both, although using only the second of these is not allowed, unless there is a means of informing the client that it has been chosen.¶

• The client, when informed of the details of the client's handling of case, has the ability to efficiently implement an appropriate case-insensitive name comparison compatible with that of the server. This includes the ability to handle mappings between single characters and multi-character strings.¶
• Implementation of case-insensitive name comparisons will typically require a case-insensitive name hash.¶

A.3. Providing Information about Server Case-Insensitive Comparisons

It is possile to provide, as part of a valid NFSv4 extension, information sufficient to allow th client to be aware of, and poentially to emulate, case-insensitive comparions implemented by the server. Such information would take the form of an OPTIONAL reaonly per-fs file atrribute. The information listed below would need to be included.¶

Whenver the value provided for a particular file system is invalid in some way, the client is justified in ignoring the attribute and acting as if it were not supported on that file system¶

• An integer denoting the version of Unicode on which the implemented case-equivalence relation was based.¶

  The value zero would be available for use to indicate that the version is not relevant, either because the file system in question is UTF8-unaware, or because there is no server procesing based on this version when the server is not case-insensitive and does not provide any normalization-related services.¶

  If the value zero is received on a case-insenstitive, the attribute value is considered invalid.¶

• Information rearding the special mapping for languages in which dot and dotless i's represent different vowel sounds (e.g. Turkish and Azeri).¶

  This could take the form of an enumrated value having the values listed below, with any other value causing the attribute to be considered invalid.¶

  • A value indicating that only the C-type mapping are to be used in handling all i characters.¶

    In the case, LATIN SMALL LETTER I (U+0069) and LATIN CAPITAL LETTER I (U+0049) are considered case-equivalent while neither LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130) nor LATIN SMALL LETTER DOTLESS I (U+0131) are considered case-equivalent to any other character.¶

  • A value indicating that only the T-type mappings are to be used in handling all i characters.¶

    In this case, LATIN SMALL LETTER DOTLESS I (U+0131) is considered case-equivalent to LATIN CAPITAL LETTER I (U+0049) while neither LATIN CAPITAL LETTER I (U+0049) nor LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130) are considered case-equivalent to any other character.¶

  • A value indicating that both C-type and T-type mappings are to be used when handling i character.¶

    This value must not be used for file system that are case-insensitive but not case-prserving.¶

    In this case, all of LATIN SMALL LETTER I (U+0069), LATIN CAPITAL LETTER I (U+0049), LATIN SMALL LETTER DOTLESS I (U+0131), and LATIN CAPITAL LETTER I WITH DOT ABOVE (U+0130) are considered case-equivalent.¶

• Handling for special and full case foldings, as described in Appendix A.2.¶

  This might take the form of a variable-length array of item of charfoldtype4, one for each character that can be subject to either S-type or F-type mappings. A possible realization of this type is described below. If this array is not of length zero and the Unicode version is zero, the attribute is considered invalid.¶

Each charfoldtype4 would contain the following:¶

• The numeric value of the UCS character, as opposed to the UTF-8 encoding of that character.¶

  If the character is one that has neither an S-type nor an F-type mapping, the attribute is considered invalid.¶

• A word with two bits, each of which indicate whether one of the two types of mapping are to be used in constryctine sets of quivalent strins, with the low-order bit referring to S-type mappings and and the next bit referring to F-type mappings. Depending on these bit settings, these mappings are either included or not in the set of case-equivalent strings associated with the partucylar character on theurrent for the file system. This in addition to any equivalences resulting from C-type mappings¶

  When either of these bits is set and the specified mapping does not exist for the associated character, the attribute is considered invalid.¶

If there are characters within the specified Unicode version that have S-type or F-type mappings specified and are not included in the array, then the equivalence set memberships for that character depend only on C-type mappings, if present.¶

A.4. Providing Information abour Server Form-Insensitive Comparisons

It is possile to provide, as part of a valid NFSv4 extension, information sufficient to allow the client to be aware of, and poentially to emulate, form-insensitive comparions implemented by the server. Such information would take the form of an OPTIONAL reaonly per-fs file atrribute. The following information would need to be included.¶

• An integer denoting the version of Unicode on which the implemented canonical equivalence was based.¶

  The value zero would be available for use to indicate that the versionis not relevant, either because the file system in question is UTF8-unaware, or because there is no server procesing based on the canonical equivalence relation.¶

• An eumerated value indicates whether names are mapped to their NFC or NFD eqivalents, or compaared in a form-insensitive manner without modification.¶

Although the attribute discussed in Appendix A.3 contins the Unicode version, allowing this one to be dispensed with, it is defined separaely for the following reasons:¶

• Because of the additional effort in defining an attribute capable of supporting case-insensitivity and the low level of interest in that feature, the Working Group might decide to define this one first.¶

• Even when they were both defined some servers might choose not to support the one only applicable to a case-insensitive environment.¶

B.1. Implementing Case-Insensitive Comparison of File Names

Implementing case-insensitive string comparisons based on equivalence classes including multi-character strings can be performed as described below. When such case-based set of equivalent strings contain multi-character strings, there are potential complexities that derive from the need to recognize such multi-character strings within the strings being compared.¶

The algorithm presented in this section requires the following for each set of equivalent strings:¶

(1):

That if there is more than one multi-character string within the set of equivalent strings, the eqivalence of those strings must be derivable from case-insensitive string equivalence using sets of equivalent strings each of whose members consist only of single-character strings.¶

(2):

That each such set contains at least one single-character string.¶

Although other sources are possible (see items EX2 and EX3 in Appendix A.1), multi-character sequences often appear in case-insensitive sets of equivalent strings as the result of the canonical decomposition of one or more precomposed characters as elements of a case-insensitive equivalence class.¶

While the algorithm presented in this section can deal with certain case-based equivalences deriving from canonical decomposition, it is not capable of providing general handling of the combination of canonical equivalence and case-based equivalence. While this can be addressed by normalizing strings before doing case-insensitive comparison, it is more efficient to do a general form-insensitive and case-insensitive string comparison in a single step as described in Appendix B.2¶

The following tables would be used by the comparison algorithm presented below.¶

• For each possible character value, the associated set of equivalent strings for case-insensitive comparison would be identified¶
• For each such set, the hash value contribution will be provided. In the case of set of equivalent stringd that do not include multi-character strings including set that only include a single (single-charater) member, this will be the hash value contribution of one particular variant (usually lower case) of the character¶
• In the case of set of equivalent string that do include multi-character strings, the hash value contribution needs to be equivalent to the combined contribution of each character within the multi-character string. In addition, for each such equivalence class, the length of the multicharacter string will be provided together with a pointer to an array describing the multi-character string, most probably presenting each character by a value of a case-seqivalent character, most probably the lower-case variant.¶

Case-insensitive comparison proceeds as follows:¶

• Implementation of case-insensitive name comparisons will typically require a case-insensitive name hash using the tables described above. If such a hash value is kept for all cached names, comparisons of hashes can be used instead of the detailed comparison set forth below. Using such hash comparisons, a large set of potentially equivalent names can be excluded based on the occurrence of hash mismatches, since case-equivalent names would have the same hash value. value.¶
• For names with matching hash values, a detailed case-insensitive comparison will be necessary. This can proceed character-by- character or byte-by-byte. However, in the byte-by-byte case, processing in the event of a mismatch must start at the start of the current character, rather than the byte at which the difference was detected.¶
• In cases in which there is a mismatch, the associated equivalence classes will be compared. When these are identical, indicating the case equivalence of the two characters, the comparison of the two strings continues at the next character of each string.¶
• When the two equivalence classes are not identical, further comparisons to determine if a single character within one string matches (except for case) a multi-character string within the other. For each of two equivalence classes being compared that include a multi-character string, the check below must be made to determine whether the multi-character string at the corresponding position of the other string being compared, is within the current equivalence class. If neither of the two equivalence classes include multi-character strings, the comparison terminates with a mismatch indication.¶
• For each equivalence class that does include a multi-character string (there might be one or two), a scan needs to be made to see of the characters at the current position if the other string matches (except for case) the multi-character string which is included in the current equivalence class. If this check succeeds, for either equivalence class, the comparison of the two strings continues at the next character of each string. In the event of failure, the same sort of comparison is done using the other current equivalence class, if it include multi-character strings. Once this check fails for all equivalence classes that include multi-character strings, the comparison terminates with a mismatch indication.¶

B.2. Form-insensitive String Comparisons

This section deals with two varieties of form-insensitive string comparison:¶

• Providing a comparison function which is form-insensitive only. For any string, whether normalized or not, this function will determine it to be equivalent to all canonically equivalent strings, including but not limited, to the normalized forms NFC and NFD¶
• Providing a comparison function which is both form-insensitive and case-insensitive. This function will determine strings that only differ in case to be equal but will also be form-insensitive, as described above.¶

The non-normative guidance provided in this Appendix is intended to be helpful in dealing with two distinct implementation areas:¶

• Implementation of server-side file systems intended to be accessed as UTF8-aware file systems using NFSv4 protocols. While it is often the case that such file systems are developed by separate organizations from those concerned with NFSv4 server development, the internationalization- related requirements specified in this document must be adhered to for successful inter-operation when using UTF8-aware file systems, making this implementation guidance apropos despite any potential organizational barriers.¶
• Implementation of NFSv4 clients that might need to provide matching internationalization-related handling for reason discussed in Section 6.3.¶

There are three basic reasons that two strings being compared might be canonically equivalent even though not identical. For each such reason, the implementation will be similar in the cases in which form-insensitive comparison (only) is being done and in which the comparison is both case-insensitive and form- insensitive.¶

• Two strings may differ only because each has a different one of two code points that are essentially the same. Three code points assigned to represent units, are essentially equivalent to the character denoting those units. For example, the OHM SIGN (U+2126) is essentially identical to the GREEK CAPITAL LETTER OMEGA (U+03A9) as MICRO SIGN (U+00B5) is to GREEK SMALL LETTER MU (U+03BC) and ANGSTROM SIGN (U+212B) is to LATIN CAPITAL LETTER A WITH RING ABOVE (U+00C5).¶

  As discussed in items EX2 and EX3 in Appendix A.1, it is possible to adjust for this situation using tables designed to resolve case-insensitive equivalence, essentially treating the unit symbols as an additional case variant, essentially ignoring the fact that the graphic representation is the same. As a result, those doing string comparisons that are both form-insensitive and case-insensitive do not need to address this issue as part of form-insensitivity, since it would be dealt with by existing case-insensitive comparison logic.¶

  Where there is no case-insensitive comparison logic, this function needs to be performed using similar tables whose primary function is to provide the decomposition of precomposed characters, as described in Appendix B.2.2.¶

• Two strings may differ in that one has the decomposed form consisting of a base character and an associated combining character while the other has a precomposed character equivalent.¶

  Although, as discussed in items EX3 in Appendix A.1, it is possible to use tables designed to resolve case-insensitive equivalence by providing as possible case-insensitively equivalent string, multi-character string providing the decomposition of precomposed characters, special logic to do so is only necessary when the decomposition is not a canonical one, i.e. it is a compatibility equivalence.¶

  In general, the table used to do comparisons, whether case-sensitive or not, needs to provide information about the canonical decomposition of precomposed characters. See Appendix B.2.2 for details.¶

• Two strings may differ in that the strings consist of combining characters that have the same effect differ as to the order in which the characters appear.¶

  There is no way this function could be performed within code primarily devoted to case-insensitive equivalence. However, this function could be added to implementations, providing both sorts of equivalence once it is determined that the base characters are case-equivalent while there is a difference of combining characters in to be resolved. (See Appendix B.2.5 for a discussion of how sets of combining characters can be compared).¶

B.3. Optimization of Form-Insensitive Comparisons

This section will discuss situations in which form-independent comparisons, for certain groups of strings, can be done in a more efficient manner than described in Appendix B.2.¶

One important group of strings is those in which all of the characters consist of a single byte. We call these strings the UTF8-onebyte subset. A string's membership in this subset can be easily determined as part of UTF8-compliance checking, hash generation, or a preliminary byte-by-byte comparison to a string whose membership status in this subset is already known.¶

As a result, there are many situations in which a form-independent string comparison can be done without reference to detailed character tables or any UTF8-to-UCS conversions. Examples follow:¶

• If the current file system is case-sensitive and either of two strings being compared are a member of the UTF8-onebyte subset the result of a byte-by-byte comparison of the two strings can be accepted as definitive without any referrence to the details of the particular canonical equivalence relation used.¶

  When neither of the strings being compared are a member of the UTF8-onebyte subset, there are further opportnities for optmized cpmarsonss, discussed below.¶

  This applies regaddless of the particular Unicode version used.¶

• If the current file system is case-insensitive and the handling of case equivalence is such that LATIN SMALL LETTER I (U+0069), and LATIN CAPITAL LETTER I (U+0049) are considered equivalent, then, when both of the strings being compared are members of UTF8-onebyte subset, a postive result for the comparson can be immediately acceoted but a negative result, need to be supplemented by simple version of case-insenstive using a 127-byte table mapping each letter to other-case equivalent. If this succeeds the strings are equivalent, while, if it does not, all the complxities of form-insensitive string comprisons need to be taken account of.¶

  This applies regaddless of the particular Unicode version used.¶

• If the current file system is case-insensitive and the handling of case equivalence is such that either LATIN SMALL LETTER I (U+0069), and LATIN CAPITAL LETTER I (U+0049) are not considered equivalent, or the handling of these characters is unknown (client only) than a variant of the above can be used.¶

  In this varient, when a byte-by-byte comparison results in a negative result, a byte-by-byte comparsion stilll needs to be done but the mapping table usedis different in that it does not map LATIN SMALL LETTER I (U+0069) and and LATIN CAPITAL LETTER I (U+0049) to each other but maps each character to itself as it does for chrater that have no case.¶

When the procudures above are not usable, further opportnities for optimized handling depend on case-sensitivity. For case-sensitive file systems, there are optimized approaches to name comparisons that can be used when either or both of the names being compared is not a member of the UTF8-onebyte subset.¶

The alternative allows a byte-by-byte comparison to be used for name comparison if at least one of the names belong to the canonical-singleton subset of strings, defined as those strings that are known to have no canonically equivalent strings. Two important facts, which implementations can take advantage of, are the following:¶

• The UTF8-onebyte subset is contained within the canonical-singleton subset.¶

  This fact can be taken advantage of when one of the two string to be compared is a member of the UTF8-onebyte subset, so no further checking is necessary in this case. As a result additional testing for memberrship in the canonical-singleton subset only needs to be done when neitther of the two strings is a member of the UTF8-onebyte subset.¶

• This set can be usefully defined without reference to the particular version of Unicode to be used. This allows this set to be used by clients in testing bames for suitability for negative name caching, as described in Appendix B.4.¶

  The set of characters can be defined as all the characters defined in a relatively early version of Unicode with certin exclusions, excluding chaacters which are the NFC form of some string, combining characters, defined as those ever present within some NFD form of a one-character string, together with OHM SIGN (U+2126).¶

  This set does not have to be changed with new Uniode versions, since, while it possible for them to add new characters to this set it is impossible to remove them since that would require converting a previously-existing character to be a combining caracter or given it a new decomposition which is impossible.¶

Implementations are likely to implement a test for strings in the canonical-singleton subset, limited to strings which are limited to strings whose UTF-8 encoding includes no character requiring more than two bytes to encode. In testing for membership in this subseet one-but character can be ignored and two-byte character need to checked against a 240-byte readonly bitmap whose bytes are likely to be available quite quickly in processor caches.¶

B.4. Restricted Client Caching to Deal with Name Equivalences

Given the name caching difficulties mentioned in Section 6.3 and the typical lack of information egarding the details many clients will want to limit name caching as described in that section. However, there might be siuations in which other approaches are desirable and we discuss the issues below:¶

• For case-sensitive file systems, name which are in the canoninical-singleton subset can effectively cached, so clients could use the full-range of name-caching techniques for such names, even the absence of detailed information about the canonical equivalence relation being used.¶

  There is overhead ovehead addedd by this ceck on the client, since, unlike the server case, there is no opportunity to combine this check with validation of UTF-8 encoding. Nevertheless, that overhad is quite small so it is likely that clients will implement it for UTF8-aware file system that are case-sensitive, rather than living with restricted name caching, as described in Section 6.3.¶

• For case-insensitive file systems, the situation is different. Even for the UTF8-onebyte subset, the possibilities of unexpected equivalence due to issues with dotted and dotless i, sharp s, and various ligatures means that simple case-based equivalences cannot be assumed.¶

  As a result, clients handling case-insensitive file systems are most likely to simply avoid potentially troublesome forms of name caching, unless full information on the equivalence relation is available. In the case that it is available, all form of name caching would be possible, but that requires the implemntation, on the client of the comparison methods described in Appendix B.2 together with the potential optimizations discussed in Appendix B.3.¶