| TOC |
|
This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”
The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.
This Internet-Draft will expire on February 16, 2010.
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document.
Existing benchmarking methodologies are based on the assumption that networking devices will drop network traffic at their performance limits. Data Center Bridging (DCB) devices, however, will attempt to throttle network endpoints before those limits are reached in order to minimize the probability of frame loss in the network. Hence, existing methodologies based around frame loss are inappropriate for DCB devices. This document takes the basic benchmarking ideas based on loss and extends them to support "lossless" Ethernet devices.
1.
Introduction
2.
Requirements
3.
General Considerations
3.1.
Test Traffic
4.
Terminology
5.
Test Setup
5.1.
Test Traffic
5.1.1.
Traffic Classification
5.1.2.
Traffic Generation
5.1.3.
Trial Duration
5.1.4.
Frame Formats
5.1.5.
Frame Measurements
5.1.6.
Frame Sizes
6.
Benchmarking Tests
6.1.
Queueput
6.1.1.
Objective
6.1.2.
Setup Parameters
6.1.3.
Procedure
6.1.4.
Measurements
6.1.5.
Reporting Format
6.2.
Maximum Forwarding Rate
6.2.1.
Objective
6.2.2.
Setup Parameters
6.2.3.
Procedure
6.2.4.
Measurements
6.2.5.
Reporting Format
6.3.
Back-off
6.3.1.
Objective
6.3.2.
Setup Parameters
6.3.3.
Procedure
6.3.4.
Measurements
6.3.5.
Reporting Format
6.4.
Back-to-Back
6.4.1.
Objective
6.4.2.
Setup Parameters
6.4.3.
Procedure
6.4.4.
Measurements
6.4.5.
Reporting Format
7.
Security Considerations
8.
IANA Considerations
9.
Normative References
Appendix A.
Acknowledgements
§
Authors' Addresses
| TOC |
This document is intended to provide a methodology for benchmarking Data Center Ethernet (DCB) switching devices that support Priority-based Flow Control (PFC) as defined in [802.1Qbb] (Institute of Electrical and Electronics Engineers, Inc., “802.1Qbb - Priority-based Flow Control,” 2009.). It extends the methodologies already defined in [RFC2544] (Bradner, S. and J. McQuaid, “Benchmarking Methodology for Network Interconnect Devices,” March 1999.) and [RFC2889] (Mandeville, R. and J. Perser, “Benchmarking Methodology for LAN Switching Devices,” August 2000.).
This RFC primarily deals with devices that use priority-based flow control (PFC), as defined in [802.1Qbb] (Institute of Electrical and Electronics Engineers, Inc., “802.1Qbb - Priority-based Flow Control,” 2009.), to actively manage the transmission rate of network endpoints in order to minimize forwarding delay and frame loss.
| TOC |
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.).
| TOC |
| TOC |
The lock-step traffic pattern, as described in section 5.1.3 of [RFC2889] (Mandeville, R. and J. Perser, “Benchmarking Methodology for LAN Switching Devices,” August 2000.), is specifically NOT required for DCB testing for two reasons:
| TOC |
As the terminology used by [RFC4689] (Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, “Terminology for Benchmarking Network-layer Traffic Control Mechanisms,” October 2006.) is specific to IP layer testing, a number of existing terms require clarification when used in the DCB benchmarking context. Additionally, a number of new terms are also presented to clarify concepts not clearly defined within the scope of [RFC4689] (Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, “Terminology for Benchmarking Network-layer Traffic Control Mechanisms,” October 2006.).
Classification: As stated in [RFC4689] (Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, “Terminology for Benchmarking Network-layer Traffic Control Mechanisms,” October 2006.), the selection of packets according to defined rules. In the context of DCB benchmarking, the Classification criterion is the value of the 802.1p priority code point field in the 802.1Q VLAN header of an Ethernet frame.
Classification Group: A collection of traffic streams that belong to a single Classification. A Conformance Vector MAY be associated with a Classification Group.
Classification Profile: The set of all Classification Groups involved in a benchmarking test.
Conformance Vector: A set of measurable stream result bounds, e.g. latency, jitter, sequencing, etc., that specify whether a frame is Conformant or Nonconformant.
Congestion Management: In the context of DCB benchmarking, Congestion Management occurs when the DUT/SUT transmits priority-based flow control (PFC) Pause frames.
Forwarding Congestion: In the context of DCB benchmarking, Forwarding Congestion is extended to include PFC pause frame transmissions from the DUT.
Intended Load: In this document, the Intended Load refers to the summation of the Intended Vectors for all Classification Groups.
Offered Load: In this document, the Offered Load refers to the summation of the Offered Vectors for all Classification Groups.
Queue Congestion: Queue congestion occurs when a DUT/SUT uses Congestion Management on a discrete set of code points. The congestion managed code points correspond to the congested queue in the DUT/SUT.
Queueput: The maximum Offered Load than can be transmitted into a DUT/SUT such that every transmitted frame matches a specific Classification rule, the DUT/SUT does NOT use priority-based flow control mechanisms to manage the ingress traffic rate, and all ingress frames are forwarded to the correct egress port. A DUT may have a different Queueput rate for each configured Classification.
| TOC |
This document extends the general test setup described in section 3 of [RFC2889] (Mandeville, R. and J. Perser, “Benchmarking Methodology for LAN Switching Devices,” August 2000.) and section 6 of [RFC2544] (Bradner, S. and J. McQuaid, “Benchmarking Methodology for Network Interconnect Devices,” March 1999.) to the benchmarking of Data Center Ethernet switching devices. [RFC2889] (Mandeville, R. and J. Perser, “Benchmarking Methodology for LAN Switching Devices,” August 2000.) and [RFC2544] (Bradner, S. and J. McQuaid, “Benchmarking Methodology for Network Interconnect Devices,” March 1999.) describe benchmarking methodologies for networking devices that intentionally drop frames at their performance limits. In DCB networks, the DUT will transmit PFC Pause frames as a Congestion Management method to throttle network endpoints, thus minimizing the probability of frame loss in the network.
| TOC |
| TOC |
Since DCB devices are expected to support multiple traffic Classifications, it is RECOMMENDED to benchmark DCB devices with multiple Classification Groups.
| TOC |
When generating multiple traffic classifications from the same test port, traffic from each class MUST be interleaved in a round-robin fashion.
| TOC |
The recommended trial duration is 300 seconds, however other durations MAY be used. Additionally, a running trial MAY be aborted once the test tool can tell that the currently running trial has failed. Two examples would be if a frame does not conform to the Conformance Vector of that frame's Classification Group, or if the test tool detects loss on a lossless queue.
| TOC |
This testing document does not mandate the use of any particular frame format for testing. Any frame that can be legally forwarded by the DUT/SUT MAY be used provided that the test instrument can make the following distinctions for each frame:
| TOC |
Packet Conformance MUST be determined for each and every test frame. The method specified for measuring Latency in [RFC2544] (Bradner, S. and J. McQuaid, “Benchmarking Methodology for Network Interconnect Devices,” March 1999.), e.g. measuring the latency of a single test frame in a traffic flow, is unsuitable for DCB benchmarking.
| TOC |
Multiple methods exist for measuring the time it takes a test frame to be forwarded by a DUT. However, both of the methods discussed in [RFC1242] (Bradner, S., “Benchmarking terminology for network interconnection devices,” July 1991.) are unsuitable for testing DCB devices, as many DCB devices alternate between both "store and forwarding" and "bit forwarding" behavior depending upon their queue congestion state. Hence, the only recommended method for measuring the time it takes a DUT to forward a test frame is "Forwarding Delay" as described in [RFC4689] (Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, “Terminology for Benchmarking Network-layer Traffic Control Mechanisms,” October 2006.).
| TOC |
| TOC |
The recommended frame sizes for Ethernet testing are 64, 128, 256, 512, 1024, 1280, 1518, 4096, 8192, and 9216 as per [RFC5180] (Popoviciu, C., Hamza, A., Van de Velde, G., and D. Dugatkin, “IPv6 Benchmarking Methodology for Network Interconnect Devices,” May 2008.). Note that this frame size includes the Ethernet CRC. This frame size MUST include a VLAN header, since [802.1Qbb] (Institute of Electrical and Electronics Engineers, Inc., “802.1Qbb - Priority-based Flow Control,” 2009.) requires the use of VLAN priority code points to distinguish different Classification Groups.
| TOC |
FCoE test traffic introduces a number of frame size constraints that make the default frame sizes specified in [RFC5180] (Popoviciu, C., Hamza, A., Van de Velde, G., and D. Dugatkin, “IPv6 Benchmarking Methodology for Network Interconnect Devices,” May 2008.) unusable:
Due to these constraints, the recommended frame sizes for FCoE testing are 128, 256, 512, 1024, 1280, 1520, 2176, and the smallest FCoE frame size supported by the test tool. This frame size MUST include a VLAN header, since [802.1Qbb] (Institute of Electrical and Electronics Engineers, Inc., “802.1Qbb - Priority-based Flow Control,” 2009.) requires the use of VLAN priority code points to distinguish different Classification Groups.
| TOC |
| TOC |
| TOC |
To determine the Queueput for one or more Traffic Classifications for a DUT using priority-based flow control.
| TOC |
The following parameters MUST be defined. Each variable is configured with the following considerations.
Each Classification Group MUST be listed. For each classification group, the following parameters MUST be specified:
Codepoint - For DCB tests, the codepoint is the VLAN priority.
Frame Size - The frame size includes both the CRC and VLAN header. See Section 5.1.6 (Frame Sizes)x for recommended frame sizes.
Burst Size - The burst size specifies the number of frames transmitted with the minimum legal IFG before pausing. A burst size of 1 specifies a constant load.
Intended Vector - The intended vector SHOULD specify the intended rate of test traffic specified as a percentage of port load.
Conformance Vector - The conformance vector is optional, but must be defined if used.
Priority-based flow control - PFC mechanisms MUST be enabled.
Background Traffic - Background traffic MAY be present.
| TOC |
A search algorithm is used to determine the Queueput for each Classification Group. If Queue Congestion is detected for a Classification Group during a trial, then the Intended Vector for the Classification Group MUST be reduced for the subsequent trial. If a Conformance Vector is specified for the test and nonconformant frames are received during a trial, then the Intended Vector SHOULD be reduced for the subsequent trial. The algorithm MUST adjust the Intended Vector for each Classification Group. The search algorithms for each Classification Group MAY be run in parallel. The test continues until all Classification Groups in the test have converged on a discrete Queueput value.
| TOC |
The Queueput for each Classification SHOULD be reported in either frames or bits per second.
If a Conformance Vector is specified for a Classification Group, the percentage of conformant frames SHOULD be reported.
The number of PFC pause frames for each code point in the Codepoint Set MUST be reported.
| TOC |
TBD
| TOC |
| TOC |
To determine the maximum forwarding rate of one or more PFC queues on a PFC capable DUT.
| TOC |
Maximum Forwarding Rate is conceptually similar to the measurement in [RFC2285] (Mandeville, R., “Benchmarking Terminology for LAN Switching Devices,” February 1998.) but works on a per-Classification basis in a DCB context. The following parameters MUST be defined. Each variable is configured with the following considerations.
Each Classification Group MUST be listed. For each classification group, the following parameters MUST be specified:
Codepoint - For DCB tests, the codepoint is the VLAN priority.
Frame Size - The frame size includes both the CRC and VLAN header. See Section 5.1.6 (Frame Sizes) for recommended frame sizes.
Burst Size - The burst size specifies the number of frames transmitted with the minimum legal IFG before pausing. A burst size of 1 specifies a constant load.
Intended Vector - The intended vector includes the intended rate of test traffic specified as a percentage of port load.
Conformance Vector - The conformance vector includes all QoS bounds required for this traffic group.
Priority-based flow control - priority-based flow control mechanisms MAY be enabled or disabled.
Background Traffic - Background traffic MAY be present.
| TOC |
The tester should iterate across all configured permutations of frame size, burst size, and Intended Vector for all Classification Groups.
| TOC |
The forwarding rate of each Classification Group SHOULD be reported as the number of test frames per second or bits per second the DUT correctly forwards to the proper egress port.
The maximum forwarding rate for each Classification Group MUST be reported as the highest recorded forwarding rate from the set of all iterations.
Both the Intended and Offered Vector of each Classification Group MUST be reported.
The number of PFC Pause frames received for each Classification MUST be reported.
If a Conformance Vector is specified for a Classification Group, the percentage of conformant frames SHOULD be reported.
| TOC |
TBD
| TOC |
| TOC |
To determine the delta between the maximum forwarding rate of a DUT and the point where the DUT ceases to use PFC to manage priority queues.
| TOC |
The following parameters MUST be defined. Each variable is configured with the following considerations.
Each Classification Group MUST be listed. For each classification group, the following parameters MUST be specified:
Codepoint - For DCB tests, the codepoint is the VLAN priority.
Frame Size - The frame size includes both the CRC and VLAN header. See Section 5.1.6 (Frame Sizes) for recommended frame sizes.
Burst Size - The burst size specifies the number of frames transmitted with the minimum legal IFG before pausing. A burst size of 1 specifies a constant load.
Intended Vector - The intended vector includes the intended rate of test traffic specified as a percentage of port load.
Conformance Vector - The conformance vector includes all QoS bounds required for this traffic group.
Priority-based flow control - priority-based flow control mechanisms MUST be enabled.
Backoff method - The recommended backoff method is to reduce the aggregate traffic load by a fixed amount while still maintaining a fixed load ratio between all Classification Groups.
| TOC |
The initial trial SHOULD begin with an Intended Load equal or greater than the Maximum Forwarding Rate of the DUT/SUT. For each subsequent trial, the aggregate load is reduced until the DUT is observed to complete a trial without activating any Congestion Management methods.
| TOC |
The Intended and Offered Vector for each Classification Group MUST be reported.
The number of PFC Pause frames received for each Classification Group MUST be reported.
If a Conformance Vector is specified for a Classification Group, the percentage of conformant frames SHOULD be reported.
| TOC |
TBD
| TOC |
| TOC |
To determine the maximum burst size a DUT handle on one or more PFC queues.
| TOC |
The following parameters MUST be defined. Each variable is configured with the following considerations
Each Classification Group MUST be listed. For each classification group, the following parameters MUST be specified:
Codepoint - For DCB tests, the codepoint is the VLAN priority.
Frame Size - The frame size includes both the CRC and VLAN header. See Section 5.1.6 (Frame Sizes) for recommended frame sizes.
Burst Size - The burst size specifies the number of frames transmitted with the minimum legal IFG before pausing. A burst size of 1 specifies a constant load.
Intended Vector - The intended vector includes the intended rate of test traffic specified as a percentage of port load.
Conformance Vector - The conformance vector includes all QoS bounds required for this traffic group.
Priority-based flow control - priority-based flow control mechanisms MAY be enabled or disabled.
The sum of all Intended Vectors on a transmitting port SHOULD equal the Maximum Offered Load (MOL) of that port.
| TOC |
A search algorithm is used to determine the maximum duration in seconds for which the configured Classification Profile can be forwarded by the DUT without active Congestion Management. If Congestion Management is detected during an iteration, then the duration MUST be reduced for the next iteration.
| TOC |
The Intended and Offered Vector for each Classification Group MUST be reported.
The number of PFC Pause frames received for each Classification Group MUST be reported.
If a Conformance Vector is specified for a Classification Group, the percentage of conformant frames SHOULD be reported.
| TOC |
TBD
| TOC |
A broken PFC implementation could lead to hypothetical denial-of-service attacks where a DUT ignores PFC pause frames and incorrectly forwards traffic as a result, or conversely throttles traffic even though no congestion condition exists. However, the test networks described in Benchmarking Methodology Working Group documents generally SHOULD NOT be reachable by anyone other than the tester(s). The security implications of such incorrect PFC handling are beyond the scope of this performance-measurement document, which is intended for testing purposes in controlled lab conditions.
| TOC |
Testers SHOULD use network addresses assigned by IANA for the purpose of testing networks.
| TOC |
| TOC |
| TOC |
| Timmons C. Player | |
| Spirent Communications | |
| Email: | timmons.player@spirent.com |
| David Newman | |
| Network Test | |
| Email: | dnewman@networktest.com |