Selective Copy-Editing Experiment for RFCs
draft-eronen-rfc-selective-experiment-00.txt

Status of this Memo

By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on September 2, 2006.

Abstract

This document proposes an experiment for the IETF RFC publication process. The experiment is limited in scope and duration. The specific experiment has been chosen because (a) it has potential to provide a significant process improvement, (b) it can be executed at a low cost, (c) it addresses a widely recognized problem in the IETF process, and (d) tool support for the experiment can be (and has been) built for it. The experiment relates to the copyediting and other manual tasks in the publication process. Specifically, the amount of work these manual tasks require differs widely between drafts, and that for a certain subset of drafts, there are either very minor editorial changes or no changes at all, modulo the different formatting requirements between RFCs and drafts. The experiment involves identification of this subset of drafts and processing them separately with a "fast path" process that uses almost entirely automated processes. For the drafts that belong to this subset, it is expected that the RFC Editor queuing time is reduced from months or years to weeks.

Table of Contents

1.  Introduction
2.  Proposed Process
3.  Process Expirement
4.  Discussion
    4.1.  Incentives
    4.2.  Document Quality
    4.3.  IESG Workload
    4.4.  Appeals
5.  IANA Considerations
6.  Security Considerations
7.  Acknowledgments
8.  References
    8.1.  Normative References
    8.2.  Informative References
§  Authors' Addresses
§  Intellectual Property and Copyright Statements

1.  Introduction

When a document is approved by the IESG, it is sent to the RFC Editor for publishing. The publication process includes processing at IANA, copyediting for grammar and spelling purposes, waiting for references to be published, the formatting of the document in the RFC style, final checks by authors in so called "Author's 48 Hours", and the storage of the draft and metainformation related to it in the RFC Editor's information systems.

Unfortunately, the entire process can take a long time, from months to over a year. There are multiple reasons behind such long processing delays:

• RFC Editor or IANA workload. This includes the workload of their subcontractors, such as the professional copyediting services sometimes employed by the RFC Editor.
• Waiting for normative references to progress through the IETF and RFC publication processes.
• Busy, unresponsive or hard-to-reach authors that do not respond in AUTH48 or do not participate in discussions with IANA.
• Events happening in parallel elsewhere that cause changes to be considered for the documents in question. This leads to longer AUTH48 periods, going back to the WG or ADs to confirm changes, and sometimes even pulling the documents back from the queue.
• Imperfect administrative processes, tool support, or lack of input materials. Documents have been known to fall between the cracks. The publication process itself is not fully automatic, and sometimes there RFC Editor has to do extra work when input material such as XML source is missing.
• Bad document quality which results in more copyediting and sometimes leads to a need to make even technical modifications.

Overall, the current delays are problematic for the IETF. The contributors expect to be able to ship products based on RFCs as soon as the specifications are approved by the IESG. The long wait after the approval delays the deployment of Internet technology, is not motivating for the participants, and brings uncertaintity that can be harmful. Long processing times increase the likelihood of events that prompt people to request "bis" level changes in AUTH48, due to implementation experience, for instance.

If the entire process for creating new specifications is lengthy, it can become a barrier to standardizing new technology. An efficient IETF process serves the needs of the Internet community best. At a high level, the process consists of various parts, such as chartering and processing at WG, IESG, and RFC Editor. All of these parts take a significant amount of time, and need to be addressed separately if a more efficient process is desired.

This document proposes a limited experiment to be conducted for the RFC publication process part, with an expectation to provide a significant improvement in the processing time for a subset of drafts. It is expected that the experiment does not cause significant extra work load either in the IETF or for the RFC Editor; tool support for this experiment is released simultaneously with the publication of this proposal.

The experiment relates to the copyediting and other manual tasks in the RFC publication process. Specifically, it has been observed that the amount of work these manual tasks require differs widely between drafts, and that for a certain subset of drafts, there are either very minor editorial changes or no changes at all, modulo the different formatting requirements between RFCs and drafts. The experiment involves identification of this subset of drafts under IETF control, and processing them separately with a "fast path" process that uses almost entirely automated processes.

The rest of this document is structed as follows. Section 2 (Proposed Process) describes the proposed process, Section 3 (Process Expirement) describes the experiment to employ it, and Section 4 (Discussion) discusses the some of the implications of this experiment as well as some potential future enhancements, should the experiment prove successful..

2.  Proposed Process

Under the proposed process, in certain cases the IETF (not the RFC Editor) makes the decision how much copyediting the document should get. The goal of this change is to focus the limited copyediting resources to those documents which would benefit most from it, and to allow documents with "good enough" language to proceed directly to publication.

In order for these documents to benefit from having to undergo less copyediting, the process focuses only on a very restricted subset of documents, namely those that do not require any special manual operations other than copyediting. A document is eligible for the new process if it fulfills the following conditions when it is approved:

• It is an IETF document that undergoes normal IETF last call and IESG evaluation.
• It has an IANA section indicating there are no IANA actions.
• It has no Internet Drafts as normative references.
• It receives no IESG or RFC Editor notes during the IESG process.
• It has been generated using XML2RFC so that automatic processing becomes possible.

The fullfillment of these conditions is either already checked during the existing process or it can be determined programmatically. Our preliminary analysis suggests that more than 20% of all current Working Group drafts meet these criteria.

An eligible document is included in this experiment if the IESG decides that copyediting would not significantly improve the quality of the document. This determination of having "good enough" language is a human decision, made by the IESG during the course of their technical review. Since the area directors often read the drafts for the first time during IESG evaluation, they will get a fairly good impression how difficult the document was to read for someone who has not seen it before. Thus, we believe IESG is a good place to make this decision. Furthermore, given that the IESG has to read the document in any case, this check does not represent a major increase of workload for the IESG. Note that the IESG will only record their impression of the language quality. It would not be a good use of the IESG to ask them to write down the observed problems or suggest improvements.

Under the new process, IESG members voting "Yes" for a given document MUST provide and others MAY provide an indication of whether the document has sufficiently good language. When providing this indication, the IESG members consider the following aspects:

• Does the document contain more than a minor number of typos, grammar mistakes, or unnecessarily difficult-to-understand language?
• Would copyediting, done by a person who is not familiar with the technical content, significantly improve the document? Or are the problems in aspects such as overall document organization or bad protocol design that cannot be realistically improved by such copyediting?
• What is the purpose of the document? For instance, fine-tuning the language is perhaps less important for a document archiving a design discussion, and more important for a protocol specification that will be read by a large number of people implementing and using the protocol.
• What is the intended status (proposed standard, informational, etc.) of the document?

An eligible document that has received solely "good enough" indications from the IESG is chosen for the fast track process. The RFC Editor determines eligibility using a tool, and inspects the IESG language quality decision. The fast track process, if chosen, eliminates the IANA, copyediting, reference wait, manual format conversion, and AUTH48 steps. The following steps are followed:

RFC Editor Note

A special RFC Editor Note is attached to the approval decision. This note indicates to the RFC Editor what level of copyediting is believed to be necessary.

Acquiring XML source

As is already done currently, the RFC Editor asks for the authors of recently approved RFCs for the XML source of the draft. If such source is not available, the regular process is followed.

Determine eligibility

The RFC Editor determines the eligibility of draft, for instance by employing a new tool to check reference status and other requirements necessary for the new process.

Format conversion

An option given to a new version of the XML2RFC tool enables it to generate output suitable for an RFC. This tool is used to generate the final RFC output.

Storage

The RFC Editor updates their information systems (such as the database of RFCs) with the new RFC and its metadata.

This process is done at the document approval notice reception / author response reception time. The RFC Editor acts often immediately after the approval notice is received, so the fast track process has potential to publish the RFC within days of the approval.

3.  Process Expirement

We believe the crucial question to answer is "Does the publication process work better with the modification proposed in this document, or without it?"

If the answer to this question is found to be "yes", then this change should be done, independently of other, more ambitious projects such as determining the overall requirements for technical publication services [TechPub] (Mankin, A. and S. Hayes, “Requirements for IETF Technical Publication Service,” October 2005.). However, an experiment is needed to better evaluate the effects of the proposed process.

The experiment needs to have a limited scope and duration. The scope of the experiment is naturally limited by the eligibility rules, so it is suggested that for a duration of one year, all drafts satisfying the eligibility and language quality rules will be run through the new process. The experiment is set to begin at the time this document is approved for publication.

Based on our initial analysis, we expect that roughly 100 documents could be eligible; depending on the quality of these documents, we expect somewhere between 25 and 50 documents to use the fast track process. The figure depends highly on how strong incentives this experiments creates for the authors to improve the language before IESG approval.

During the experiment, the RFC Editor collects separate statistics of the processing and queuing times for the regular and fast track processes. A person designated as the experiment supervisor posts feedback forms to document authors and WG chairs with documents going through the new process. At the end of the experiment feedback is also solicited from the IESG.

An evaluation is performed the end of the experiment and the results are published as an Internet Draft. The evaluation involves employing the statistics to determine the effect of the fast track process on document processing time and effort at the RFC Editor organization and summary of the feedback received.

4.  Discussion

This change allows the IESG to focus the limited copyediting resources to documents where the benefits are the largest. It is expected that the reduced workload for the subset of documents that go through this process will also reduce the processing time of other documents, given the same level of resources devoted to the RFC Editor activities.

In making the copyediting level decision, the IESG is in a better position than the RFC editor to consider all the factors involved (e.g., purpose of the document, priority). We believe it is in the interest of IETF community to make this decision transparently. When this decision is recorded in the public records that IESG makes available, this condition is fulfilled.

More fine-grained scheme that goes beyond on/off control of the copyediting can also be considered. However, it is suggested that such a scheme be considered if the results of the experiment prove useful.

We have not yet fully analyzed what other choices exist for making this decision than IESG evaluation. Some other alternatives appear to be possible as well. For instance, there are a number of review boards such as GEN-ART that could also provide input.

4.1.  Incentives

We believe this arrangement would better align the incentives of various parties with the IETF's goals.

Specifically, this process ensures that the IETF has control over the level of copy editing. If the RFC Editor function is contracted to a for-profit entity, that entity has an incentive to increase the amount of copyediting, and ask for more funding. In a true market competition between service providers would control this, but we do not currently believe there will be significant price competition for the RFC Editor contract.

The experiment also provides better incentives for document authors, WG chairs, and other reviewers. If better-quality text means the document progresses faster, people interested in the document have an incentive to fix the text earlier (e.g., provide more editorial comments during WG last call). These people are also in good position to know what changes are purely editorial and what actually change the meaning of the text.

4.2.  Document Quality

One argument that could be made against this experiment is that less involvement by the RFC editor means that quality will suffer.

We believe the experiment will have exactly the opposite effect. The editing work done by the RFC editor does very little to increase the quality of documents that are already in a pretty good shape. This experiment allows focusing the limited resources on those documents where the "return on investment" is the largest. It also creates incentives for the authors to work on the language before the document is approved.

Another potential complication is the difference between the XML2RFC output and the style currently used by the RFC editor. Discussions about the exact formatting requirements have been going on since November 2005, and when used with the "rfcedstyle" option, version 1.31pre4 produces output that is believed to match the current RFC editor style. While it is possible that some differences remain, and that the preferred style changes over time, we believe the current formatting is more than acceptable, and fine-tuning it further does not produce value for the IETF community.

4.3.  IESG Workload

The experiment intentionally proposes a very simple process for determining which documents meet the language quality criterion, as explained in Section 2 (Proposed Process).

We expect that the IESG members can make this decision without spending any more time than they already do. We also do not expect the IESG to produce an explanation why the document was or was not chosen, list any of the language problems identified in the document, or to negotiate about the decision with the document authors.

4.4.  Appeals

One possible problem with the experiment is that [RFC2026] (Bradner, S., “The Internet Standards Process -- Revision 3,” October 1996.) specifies a two-month period for appealing IESG decisions. Some people have interpreted this to mean that no document can be published faster than this.

However, appeals to the IESG are almost always filed within days of the decision. Even in cases when writing the complete appeal text may take some time, a "notice of intention to appeal" is often given immediately. The "fast track" process is also limited to documents that go through IETF Last Call, and people who appeal the IESG decision read and comment the documents already during the last call.

Furthermore, the two-month period has not been observed rigidly recently. While the average RFC queue delay ensures that plenty of time is left for appeals, in 2005 five RFCs were published less than two months from their approval. (We would be interested in knowing how this was possible, so we could try to get our documents to this "fast track" -- the record processing time was only five days! :-).

Moreover, in the case of standards track and BCP RFCs, the IESG can always reverse its decision after the RFC has been published by downgrading the document to "Historic".

For this experiment, we suggest that documents known to be controversial should not be selected for the fast track process. Since more than 99% of documents are not appealed, this is unlikely to affect the number of documents in the experiment. In addition, it is suggested that for the documents selected for the fast track, the approval notices include a request to make the intent to appeal known within two weeks.

5.  IANA Considerations

This document has no IANA Actions.

6.  Security Considerations

This document does not introduce any new security considerations.

7.  Acknowledgments

The authors would like to thank Bob Braden, Brian Carpenter, and Stephen Hayes for ideas and interesting discussions about this problem space.

8.  References

8.1. Normative References

8.2. Informative References

Authors' Addresses

Full Copyright Statement

Copyright © The IETF Trust (2006).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an “AS IS” basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.