Network Working Group A. Morton
Internet-Draft AT&T Labs
Updates: ???? (if approved) R. Geib
Intended status: Standards Track Deutsche Telekom
Expires: February 15, 2021 L. Ciavattone
AT&T Labs
August 14, 2020

Metrics and Methods for IP Capacity
draft-ietf-ippm-capacity-metric-method-03

Abstract

This memo revisits the problem of Network Capacity metrics first examined in RFC 5136. The memo specifies a more practical Maximum IP-layer Capacity metric definition catering for measurement purposes, and outlines the corresponding methods of measurement.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on February 15, 2021.

Copyright Notice

Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.


Table of Contents

1. Introduction

The IETF's efforts to define Network and Bulk Transport Capacity have been chartered and progressed for over twenty years. Over that time, the performance community has seen development of Informative definitions in [RFC3148] for Framework for Bulk Transport Capacity (BTC), RFC 5136 for Network Capacity and Maximum IP-layer Capacity, and the Experimental metric definitions and methods in [RFC8337], Model-Based Metrics for BTC.

This memo revisits the problem of Network Capacity metrics examined first in [RFC3148] and later in [RFC5136]. Maximum IP-Layer Capacity and [RFC3148] Bulk Transfer Capacity (goodput) are different metrics. Max IP-layer Capacity is like the theoretical goal for goodput. There are many metrics in [RFC5136], such as Available Capacity. Measurements depend on the network path under test and the use case. Here, the main use case is to assess the maximum capacity of the access network, with specific performance criteria used in the measurement.

This memo recognizes the importance of a definition of a Maximum IP-layer Capacity Metric at a time when access speeds have increased dramatically; a definition that is both practical and effective for the performance community's needs, including Internet users. The metric definition is intended to use Active Methods of Measurement [RFC7799], and a method of measurement is included.

The most direct active measurement of IP-layer Capacity would use IP packets, but in practice a transport header is needed to traverse address and port translators. UDP offers the most direct assessment possibility, and in the [copycat][copycat] measurement study to investigate whether UDP is viable as a general Internet transport protocol, the authors found that a high percentage of paths tested support UDP transport. A number of liaisons have been exchanged on this topic [ refs to ITU-T SG12, ETSI STQ, BBF liaisons ], discussing the laboratory and field tests that support the UDP-based approach to IP-layer Capacity measurement.

This memo also recognizes the many updates to the IP Performance Metrics Framework [RFC2330] published over twenty years, and makes use of [RFC7312] for Advanced Stream and Sampling Framework, and RFC 8468 [RFC8468]IPv4, IPv6, and IPv4-IPv6 Coexistence Updates.

1.1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14[RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

2. Scope and Goals

The scope of this memo is to define a metric and corresponding method to unambiguously perform Active measurements of Maximum IP-Layer Capacity, along with related metrics and methods.

The main goal is to harmonize the specified metric and method across the industry, and this memo is the vehicle through which working group (and eventually, IETF) consensus will be captured and communicated to achieve broad agreement, and possibly result in changes in the specifications of other Standards Development Organizations (SDO) (through the SDO's normal contribution process, or through liaison exchange).

A local goal is to aid efficient test procedures where possible, and to recommend reporting with additional interpretation of the results. Also, to foster the development of protocol support for this metric and method of measurement (all active testing protocols currently defined by the IPPM WG are UDP-based, meeting a key requirement of these methods). The supporting protocol development to measure this metric according to the specified method is a key future contribution to Internet measurement.

3. Motivation

As with any problem that has been worked for many years in various SDOs without any special attempts at coordination, various solutions for metrics and methods have emerged.

There are five factors that have changed (or begun to change) in the 2013-2019 time frame, and the presence of any one of them on the path requires features in the measurement design to account for the changes:

  1. Internet access is no longer the bottleneck for many users.
  2. Both speed and latency are important to user's satisfaction.
  3. UDP's growing role in Transport, in areas where TCP once dominated.
  4. Content and applications moving physically closer to users.
  5. Less emphasis on ISP gateway measurements, possibly due to less traffic crossing ISP gateways in future.

4. General Parameters and Definitions

This section lists the REQUIRED input factors to specify a Sender or Receiver metric.

A non-Parameter which is required for several metrics is defined below:

Note that time stamps, sequnce numbers, etc. will be established by the test protocol.

5. IP-Layer Capacity Singleton Metric Definitions

This section sets requirements for the following components to support the Maximum IP-layer Capacity Metric.

5.1. Formal Name

Type-P-IP-Capacity, or informally called IP-layer Capacity.

Note that Type-P depends on the chosen method.

5.2. Parameters

This section lists the REQUIRED input factors to specify the metric, beyond those listed in Section 4.

No additional Parameters are needed.

5.3. Metric Definitions

This section defines the REQUIRED aspects of the measureable IP-layer Capacity metric (unless otherwise indicated) for measurements between specified Source and Destination hosts:

Define the IP-layer capacity, C(T,I,PM), to be the number of IP-layer bits (including header and data fields) in packets that can be transmitted from the Src host and correctly received by the Dst host during one contiguous sub-interval, dt.

The number of these IP-layer bits is designated n0[dtn,dtn+1] for a specific dt.

When the packet size is known and of fixed size, the packet count during a single sub-interval dt multiplied by the total bits in IP header and data fields is equal to n0[dtn,dtn+1].

Anticipating a Sample of Singletons, the interval dt SHOULD be set to a natural number m so that T+I = T + m*dt with dtn+1 - dtn = dt and with 1 <= n <= m.

Parameter PM represents other performance metrics [see section Related Round-Trip Delay and Loss Definitions below]; their measurement results SHALL be collected during measurement of IP-layer Capacity and associated with the corresponding dtn for further evaluation and reporting.

Mathematically, this definition can be represented as:

                        ( n0[dtn,dtn+1] )
        C(T,I,PM) = -------------------------
                               dt

Equation for IP-Layer Capacity

Measurements according to these definitions SHALL use UDP transport layer.

5.4. Related Round-Trip Delay and Loss Definitions

RTD[dtn-1,dtn] is defined as a sample of the [RFC2681] Round-trip Delay between the Src host and the Dst host over the interval [T,T+I]. The statistics used to to summarize RTD[dtn-1,dtn] MAY include the minimum, maximum, and mean.

RTL[dtn-1,dtn] is defined as a sample of the [RFC6673] Round-trip Loss between the Src host and the Dst host over the interval [T,T+I]. The statistics used to to summarize RTL[dtn-1,dtn] MAY include the lost packet count and the lost packet ratio.

5.5. Discussion

See the corresponding section for Maximum IP-Layer Capacity.

5.6. Reporting the Metric

The IP-Layer Capacity SHALL be reported with meaningful resolution, in units of Megabits per second (Mbps).

The Related Round Trip Delay and/or Loss metric measurements for the same Singleton SHALL be reported, also with meaningful resolution for the values measured.

Individual Capacity measurements MAY be reported in a manner consistent with the Maximum IP-Layer Capacity, see Section 9.

6. Maximum IP-Layer Capacity Metric Definitions (Statistic)

This section sets requirements for the following components to support the Maximum IP-layer Capacity Metric.

6.1. Formal Name

Type-P-Max-IP-Capacity, or informally called Maximum IP-layer Capacity.

Note that Type-P depends on the chosen method.

6.2. Parameters

This section lists the REQUIRED input factors to specify the metric, beyond those listed in Section 4.

No additional Parameters or definitions are needed.

6.3. Metric Definitions

This section defines the REQUIRED aspects of the Maximum IP-layer Capacity metric (unless otherwise indicated) for measurements between specified Source and Destination hosts:

Define the Maximum IP-layer capacity, Maximum_C(T,I,PM), to be the maximum number of IP-layer bits n0[dtn,dtn+1] that can be transmitted in packets from the Src host and correctly received by the Dst host, over all dt length intervals in [T, T+I], and meeting the PM criteria. Equivalently the Maximum of a Sample of size m of C(T,I,PM) collected during the interval [T, T+I] and meeting the PM criteria.

The interval dt SHOULD be set to a natural number m so that T+I = T + m*dt with dtn+1 - dtn = dt and with 1 <= n <= m.

Parameter PM represents the other performance metrics [see section Related Round-Trip Delay and Loss Definitions below] and their measurement results for the maximum IP-layer capacity. At least one target performance threshold (PM criterion) MUST be defined. If more than one target performance threshold is defined, then the sub-interval with maximum number of bits transmitted MUST meet all the target performance thresholds.

Mathematically, this definition can be represented as:

                        max  ( n0[dtn,dtn+1] )
                       [T,T+I]
  Maximum_C(T,I,PM) = -------------------------
                                 dt
 where:
    T                                      T+I
    _________________________________________
    |   |   |   |   |   |   |   |   |   |   |
dtn=1   2   3   4   5   6   7   8   9  10  n+1
                                       n=m

Equation for Maximum Capacity

In this definition, the m sub-intervals can be viewed as trials when the Src host varies the transmitted packet rate, searching for the maximum n0 that meets the PM criteria measured at the Dst host in a test of duration, I. When the transmitted packet rate is held constant at the Src host, the m sub-intervals may also be viewed as trials to evaluate the stability of n0 and metric(s) in the PM list over all dt-length intervals in I.

Measurements according to these definitions SHALL use UDP transport layer.

6.4. Related Round-Trip Delay and Loss Definitions

RTD[dtn,dtn+1] is defined as a sample of the [RFC2681] Round-trip Delay between the Src host and the Dst host over the interval [T,T+I], and corresponds to the dt interval containing Maximum_C(T,I,PM). The statistics used to to summarize RTD[dtn,dtn+1] MAY include the minimum, maximum, and mean.

RTL[dtn,dtn+1] is defined as a sample of the [RFC6673] Round-trip Loss between the Src host and the Dst host over the interval [T,T+I] and corresponds to the dt interval containing Maximum_C(T,I,PM). The statistics used to to summarize RTL[dtn-1,dtn] MAY include the lost packet count and the lost packet ratio.

6.5. Discussion

If traffic conditioning applies along a path for which Maximum _C(T,I,PM) is to be determined, different values for dt SHOULD be picked and measurements be executed during multiple intervals [T, T+I]. Any single interval dt SHOULD be chosen so that is an integer multiple of increasing values k times serialisation delay of a path MTU at the physical interface speed where traffic conditioning is expected. This should avoid taking configured burst tolerance singletons as a valid Maximum _C(T,I,PM) result.

A Maximum_C(T,I,PM) without any indication of bottleneck congestion, be that an increasing latency, packet loss or ECN marks during a measurement interval I, is likely to underestimate Maximum_C(T,I,PM).

6.6. Reporting the Metric

The Maximum IP-Layer Capacity SHALL be reported with meaningful resolution, in units of Megabits per second.

The Related Round Trip Delay and/or Loss metric measurements for the same Singleton SHALL be reported, also with meaningful resolution for the values measured.

When there are demonstrated and repeatable Capacity modes in the Sample, then the Maximum IP-Layer Capacity SHALL be reported for each mode, along with the relative time from the beginning of the stream that the mode was observed to be present. Bimodal Maxima have been observed with some services, sometimes called a "turbo mode" intending to deliver short transfers more quickly, or reduce the initial buffering time for some video streams. Note that modes lasting less than dt duration will not be detected.

Some transmission technologies have multiple methods of operation that may be activated when channel conditions degrade or improve, and these transmission methods may determine the Maximum IP-Layer Capacity. Examples include line-of-sight microwave modulator constellations, or cellular modem technologies where the changes may be initiated by a user moving from one coverage area to another. Operation in the different transmission methods may be observed over time, but the modes of Maximum IP-Layer Capacity will not be activated deterministically as with the "turbo mode" described in the paragraph above.

7. IP-Layer Sender Bit Rate Singleton Metric Definitions

This section sets requirements for the following components to support the IP-layer Sender Bitrate Metric.

7.1. Formal Name

Type-P-IP-Sender-Bit-Rate, or informally called IP-layer Sender Bitrate.

Note that Type-P depends on the chosen method.

7.2. Parameters

This section lists the REQUIRED input factors to specify the metric, beyond those listed in Section 4.

S SHALL be longer than I, primarily to account for on-demand activation of the path, or any preamble to testing required.

st SHOULD be much smaller than the sub-interval dt. The st parameter does not have relevance when the Source is transmitting at a fixed rate throughout S.

7.3. Metric Definition

This section defines the REQUIRED aspects of the IP-layer Sender Bitrate metric (unless otherwise indicated) for measurements at the specified Source on packets addressed for the intended Destination host and matching the required Type-P:

Define the IP-layer Sender Bit Rate, B(S,st), to be the number of IP-layer bits (including header and data fields) that are transmitted from the Source during one contiguous sub-interval, st, during the test interval S (where S SHALL be longer than I), and where the fixed-size packet count during that single sub-interval st also provides the number of IP-layer bits in any interval: n0[stn-1,stn].

Measurements according to these definitions SHALL use UDP transport layer. Any feedback from Dst host to Src host received by Src host during an interval [stn-1,stn] MUST NOT result in an adaptation of the Src host traffic conditioning during this interval.

7.4. Discussion

Both the Sender and Receiver or (source and destination) bit rates SHOULD be assessed as part of a measurement.

7.5. Reporting the Metric

The IP-Layer Sender Bit Rate SHALL be reported with meaningful resolution, in units of Megabits per second.

Individual IP-Layer Sender Bit Rate measurements are discussed further in Section 9.

8. Method of Measurement

The duration of a test, I, MUST be constrained in a production network, since this is an active test method and it will likely cause congestion on the Src to Dst host path during a test.

Additional Test methods and configurations may be provided in this section, after review and further testing.

8.1. Load Rate Adjustment Algorithm

A table SHALL be pre-built defining all the offered load rates that will be supported (R1 – Rn, in ascending order). Each rate is defined as datagrams of size S, sent as a burst of count C, every time interval T. While it is advantageous to use datagrams of as large a size as possible, it may be prudent to use a slightly smaller maximum that allows for secondary protocol headers and/or tunneling without resulting in IP-layer fragmentation.

At the beginning of a test, the sender begins sending at rate R1 and the receiver starts a feedback timer at interval F (while awaiting inbound datagrams). As datagrams are received they are checked for sequence number anomalies (loss, out-of-order, duplication, etc.) and the delay variation is measured (one-way or round-trip). This information is accumulated until the feedback timer F expires and a status feedback message is sent from the receiver back to the sender, to communicate this information. The accumulated statistics are then reset by the receiver for the next feedback interval. As feedback messages are received back at the sender, they are evaluated to determine how to adjust the current offered load rate (Rx).

If the feedback indicates that there were no sequence number anomalies AND the delay variation was below the lower threshold, the offered load rate is increased. If congestion has not been confirmed up to this point, the offered load rate is increased by more than one rate (e.g., Rx+10). This allows the offered load to quickly reach a near-maximum rate. Conversely, if congestion has been previously confirmed, the offered load rate is only increased by one (Rx+1).

If the feedback indicates that sequence number anomalies were detected OR the delay variation was above the upper threshold, the offered load rate is decreased. If congestion is confirmed by the current feedback message being processed, the offered load rate is decreased by more than one rate (e.g., Rx-30). This one-time reduction is intended to compensate for the fast initial ramp-up. In all other cases, the offered load rate is only decreased by one (Rx-1).

If the feedback indicates that there were no sequence number anomalies AND the delay variation was above the lower threshold, but below the upper threshold, the offered load rate is not changed. This allows time for recent changes in the offered load rate to stabilize, and the feedback to represent current conditions more accurately.

Lastly, the method for confirming congestion is that there were sequence number anomalies OR the delay variation was above the upper threshold for two consecutive feedback intervals. The algorithm described above is also presented in ITU-T Rec. Y.1540, 2020 version[Y.1540], in Annexes A and B, and implemented in the reference for Section 8.4, Running Code.

8.2. Measurement Qualification or Verification

When assessing a Maximum rate as the metric specifies, artificially high (optimistic) values might be measured until some buffer on the path is filled. Other causes include bursts of back-to-back packets with idle intervals delivered by a path, while the measurement interval (dt) is small and aligned with the bursts. The artificial values might result in an un-sustainable Maximum Capacity observed when the method of measurement is searching for the Maximum, and that would not do. This situation is different from the bi-modal service rates (discussed under Reporting), which are characterized by a multi-second duration (much longer than the measured RTT) and repeatable behavior.

There are many ways that the Method of Measurement could handle this false-max issue. The default value for measurement of singletons (dt = 1 second) has proven to a be of practical value during tests of this method, allows the bimodal service rates to be characterized, and it has an obvious alignment with the reporting units (Mbps).

Another approach comes from Section 24 of RFC 2544[RFC2544] and its discussion of Trial duration, where relatively short trials conducted as part of the search are followed by longer trials to make the final determination. In the production network, measurements of singletons and samples (the terms for trials and tests of Lab Benchmarking) must be limited in duration because they may be service-affecting. But there is sufficient value in repeating a sample with a fixed sending rate determined by the previous search for the Max IP-layer Capacity, to qualify the result in terms of the other performance metrics measured at the same time.

A qualification measurement for the search result is a subsequent measurement, sending at a fixed 99.x % of the Max IP-layer Capacity for I, or an indefinite period. The same Max Capacity Metric is applied, and the Qualification for the result is a sample without packet loss or a growing minimum delay trend in subsequent singletons (or each dt of the measurement interval, I). Samples exhibiting losses or increasing queue occupation require a repeated search and/or test at reduced fixed sender rate for qualification.

Here, as with any Active Capacity test, the test duration must be kept short. 10 second tests for each direction of transmission are common today. The default measurement interval specified here is I = 10 seconds). In combination with a fast search method and user-network coordination, the concerns raised in RFC 6815[RFC6815] are alleviated. The method for assessing Max IP Capacity is different from classic [RFC2544] methods: they use short term load adjustment and are sensitive to loss and delay, like other congestion control algorithms used on the Internet every day.

8.3. Measurement Considerations

In general, the wide-spread measurements that this memo encourages will encounter wide-spread behaviors. The bimodal IP Capacity behaviors already discussed in Section 6.6 are good examples.

In general, it is RECOMMENDED to locate test endpoints as close to the intended measured link(s) as practical (this is not always possible for reasons of scale; there is a limit on number of test endpoints coming from many perspecitves, management and measurement traffic for example).

The path measured may be state-full based on many factors, and the Parameter "Time of day" when a test starts may not be enough information. Repeatable testing may require the time from the beginning of a measured flow, and how the flow is constructed including how much traffic has already been sent on that flow when a state-change is observed, because the state-change may be based on time or bytes sent or both.

Many different traffic shapers and on-demand access technologies may be encountered, as anticipated in [RFC7312], and play a key role in measurement results. Methods MUST be prepared to provide a short preamble transmission to activate on-demand access, and to discard the preamble from subsequent test results.

Conditions which might be encountered during measurement, where packet losses may occur independently from the measurement sending rate:

  1. Congestion of an interconnection or backbone interface may appear as packet losses distributed over time in the test stream, due to much higher rate interfaces in the backbone.
  2. Packet loss due to use of Random Early Detection (RED) or other active queue management.
  3. There may be only small delay variation independent of sending rate under these conditions, too.
  4. Persistent competing traffic on measurement paths that include shared media may cause random packet losses in the test stream.

It is possible to mitigate these conditions using the flexibility of the load-rate adjusting algorithm described in Section 8.1 above (tuning specific parameters).

In general, results depend on the sending stream characteristics; the measurement community has known this for a long time, and needs to keep it front of mind. Although the default is a single flow (F=1) for testing, use of multiple flows may be advantageous for the following reasons:

  1. the test hosts may be able to create higher load than with a single flow, or parallel test hosts may be used to generate 1 flow each.
  2. there may be link aggregation present (flow-based load balancing) and multiple flows are needed to occupy each member of the aggregate.
  3. access policies may limit the IP-Layer Capacity depending on the Type-P of packets, possibly reserving capacity for various stream types.

Each flow would be controlled using its own implementation of the Load Adjustment (Search) Algorithm.

As testing continues, implementers should expect some evolution in the methods. The ITU-T has published a Supplement (60) to the Y-series of Recommendations, "Interpreting ITU-T Y.1540 maximum IP-layer capacity measurements", [Y.Sup60], which is the result of continued testing with the metric and method described here.

8.4. Running Code

Much of the development of the method and comparisons with existing methods conducted at IETF Hackathons and elsewhere have been based on the example udpst Linux measurement tool (which is a working reference for further development) [udpst]. The current project:

9. Reporting Formats

The singleton IP-Layer Capacity results SHOULD be accompanied by the context under which they were measured.

The Maximum IP-Layer Capacity results SHOULD be reported in the format of a table with a row for each of the test Phases and Number of Flows. There SHOULD be columns for the phases with number of flows, and for the resultant Maximum IP-Layer Capacity results for the aggregate and each flow tested.

As mentioned in Section 6.6, bi-modal (or multi-modal) maxima SHALL be reported for each mode separately.

Maximum IP-layer Capacity Results
Phase, # Flows Max IP-Layer Capacity, Mbps Loss Ratio RTT min, max, msec
Search,1 967.31 0.0002 30, 58
Verify,1 966.00 0.0000 30, 38

Static and configuration parameters:

The sub-interval time, dt, MUST accompany a report of Maximum IP-Layer Capacity results, and the remaining Parameters from Section 4, General Parameters.

The PM list metrics corresponding to the sub-interval where the Maximum Capacity occurred MUST accompany a report of Maximum IP-Layer Capacity results, for each test phase.

The IP-Layer Sender Bit rate results SHOULD be reported in the format of a table with a row for each of the test Phases, sub-intervals (st) and Number of Flows. There SHOULD be columns for the phases with number of flows, and for the resultant IP-Layer Sender Bit rate results for the aggregate and each flow tested.

IP-layer Sender Bit Rate Results
Phase, Flow or Aggregate st, sec Sender Bit Rate, Mbps ??
Search,1 0.00 - 0.05 345 __
Search,2 0.00 - 0.05 289 __
Search,Agg 0.00 - 0.05 634 __

Static and configuration parameters:

The subinterval time, st, MUST accompany a report of Sender IP-Layer Bit Rate results.

Also, the values of the remaining Parameters from Section 4, General Parameters, MUST be reported.

9.1. Configuration and Reporting Data Formats

As a part of the multi-Standards Development Organization (SDO) harmonization of this metric and method of measurement, one of the areas where the Broadband Forum (BBF) contributed its expertise was in the definition of an information model and data model for configuration and reporting. These models are consistent with the metric parameters and default values specified as lists is this memo. [TR-471] provides the Information model that was used to prepare a full data model in related BBF work. The BBF has als carefully considered topics within its purvue, such as placement of measurement systems within the access archtecture.

10. Security Considerations

Active metrics and measurements have a long history of security considerations. The security considerations that apply to any active measurement of live paths are relevant here. See [RFC4656] and [RFC5357].

When considering privacy of those involved in measurement or those whose traffic is measured, the sensitive information available to potential observers is greatly reduced when using active techniques which are within this scope of work. Passive observations of user traffic for measurement purposes raise many privacy issues. We refer the reader to the privacy considerations described in the Large Scale Measurement of Broadband Performance (LMAP) Framework [RFC7594], which covers active and passive techniques.

There are some new considerations for Capacity measurement as described in this memo.

  1. Cooperating Source and Destination hosts and agreements to test the path between the hosts are REQUIRED.
  2. Integrity protection for feedback messages conveying measurements is RECOMMENDED.
  3. Hosts SHOULD limit the number of simultaneous tests to avoid resource exhaust and inaccuate results.
  4. Senders MUST be rate-limited. This can be accomplished using the pre-built table defining all the offered load rates that will be supported (Section 8.1). The recommended load-control search algorithm results in "ramp up" from the lowest rate in the table.
  5. Service subscribers with limited data volumes who conduct extensive capacity testing might experience the effects of Service Provider controls on their service. Testing with the Service Provider's measurement hosts SHOULD be limited in frequency and/or overall volume of test traffic.

The exact specification of these features is left for the future protocol development.

11. IANA Considerations

This memo makes no requests of IANA.

12. Acknowledgements

Thanks to Joachim Fabini, Matt Mathis, Ignacio Alvarez-Hamelin, and Wolfgang Balzer for their extensive comments on the memo and related topics.

13. References

13.1. Normative References

[RFC1242] Bradner, S., "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, DOI 10.17487/RFC1242, July 1991.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J. and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, DOI 10.17487/RFC2330, May 1998.
[RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, March 1999.
[RFC2681] Almes, G., Kalidindi, S. and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, September 1999.
[RFC2889] Mandeville, R. and J. Perser, "Benchmarking Methodology for LAN Switching Devices", RFC 2889, DOI 10.17487/RFC2889, August 2000.
[RFC3148] Mathis, M. and M. Allman, "A Framework for Defining Empirical Bulk Transfer Capacity Metrics", RFC 3148, DOI 10.17487/RFC3148, July 2001.
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J. and M. Zekauskas, "A One-way Active Measurement Protocol (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006.
[RFC5136] Chimento, P. and J. Ishac, "Defining Network Capacity", RFC 5136, DOI 10.17487/RFC5136, February 2008.
[RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G. and D. Dugatkin, "IPv6 Benchmarking Methodology for Network Interconnect Devices", RFC 5180, DOI 10.17487/RFC5180, May 2008.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K. and J. Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", RFC 5357, DOI 10.17487/RFC5357, October 2008.
[RFC6201] Asati, R., Pignataro, C., Calabria, F. and C. Olvera, "Device Reset Characterization", RFC 6201, DOI 10.17487/RFC6201, March 2011.
[RFC6412] Poretsky, S., Imhoff, B. and K. Michielsen, "Terminology for Benchmarking Link-State IGP Data-Plane Route Convergence", RFC 6412, DOI 10.17487/RFC6412, November 2011.
[RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label for Equal Cost Multipath Routing and Link Aggregation in Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011.
[RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, DOI 10.17487/RFC6673, August 2012.
[RFC6815] Bradner, S., Dubray, K., McQuaid, J. and A. Morton, "Applicability Statement for RFC 2544: Use on Production Networks Considered Harmful", RFC 6815, DOI 10.17487/RFC6815, November 2012.
[RFC6985] Morton, A., "IMIX Genome: Specification of Variable Packet Sizes for Additional Testing", RFC 6985, DOI 10.17487/RFC6985, July 2013.
[RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling Framework for IP Performance Metrics (IPPM)", RFC 7312, DOI 10.17487/RFC7312, August 2014.
[RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., Aitken, P. and A. Akhter, "A Framework for Large-Scale Measurement of Broadband Performance (LMAP)", RFC 7594, DOI 10.17487/RFC7594, September 2015.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, May 2016.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017.
[RFC8337] Mathis, M. and A. Morton, "Model-Based Metrics for Bulk Transport Capacity", RFC 8337, DOI 10.17487/RFC8337, March 2018.
[RFC8468] Morton, A., Fabini, J., Elkins, N., Ackermann, M. and V. Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for the IP Performance Metrics (IPPM) Framework", RFC 8468, DOI 10.17487/RFC8468, November 2018.

13.2. Informative References

[copycat] Edleine, K., Kuhlewind, K., Trammell, B. and B. Donnet, "copycat: Testing Differential Treatment of New Transport Protocols in the Wild (ANRW '17)", July 2017.
[RFC8239] Avramov, L. and J. Rapp, "Data Center Benchmarking Methodology", RFC 8239, DOI 10.17487/RFC8239, August 2017.
[TR-471] Morton, A., "Broadband Forum TR-471: IP Layer Capacity Metrics and Measurement", July 2020.
[TST009] Morton, R. A., "ETSI GS NFV-TST 009 V3.1.1 (2018-10), "Network Functions Virtualisation (NFV) Release 3; Testing; Specification of Networking Benchmarks and Measurement Methods for NFVI"", October 2018.
[udpst] AT&T, "UDP Speed Test Open Broadband project", August 2020.
[Y.1540] Y.1540, I. R., "Internet protocol data communication service - IP packet transfer and availability performance parameters", December 2019.
[Y.Sup60] Morton, A., "Recommendation Y.Sup60, (04/20) Interpreting ITU-T Y.1540 maximum IP-layer capacity measurements", June 2020.

Authors' Addresses

Al Morton AT&T Labs 200 Laurel Avenue South Middletown,, NJ 07748 USA Phone: +1 732 420 1571 Fax: +1 732 368 1192 EMail: acm@research.att.com
Ruediger Geib Deutsche Telekom Heinrich Hertz Str. 3-7 Darmstadt, 64295 Germany Phone: +49 6151 5812747 EMail: Ruediger.Geib@telekom.de
Len Ciavattone AT&T Labs 200 Laurel Avenue South Middletown,, NJ 07748 USA EMail: lencia@att.com