Benchmarking Terminology for Wireless LAN Switching Systems
draft-alexander-bmwg-wlan-switch-term-01

Status of this Memo

By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on July 5, 2008.

Abstract

This document provides a terminology to be used when performing performance test and benchmarking of wireless LAN (WLAN) switches and controllers, including systems comprising groups of controllers and Access Points. The various wireless-specific device nomenclature, as well as the definitions of configuration parameters and test conditions that may be used to characterize the performance of these devices, are provided. The document also defines some of the metrics used during WLAN switch benchmarking. This terminology is to be used in conjunction with the companion methodology document.

Table of Contents

1.  Introduction

2.  Existing definitions and requirements

3.  Definitions of terms
    3.1.  General terms
        3.1.1.  Wireless LAN (WLAN)
        3.1.2.  Access controller (AC)
        3.1.3.  Endstation (STA)
        3.1.4.  Wireless termination point (WTP)
        3.1.5.  Service set identifier (SSID)
        3.1.6.  Rogue device
        3.1.7.  Provisioned WTP
        3.1.8.  Unprovisioned WTP
        3.1.9.  Unicast traffic flow
        3.1.10.  Multicast traffic flow
        3.1.11.  Best-effort traffic
        3.1.12.  High-priority traffic
        3.1.13.  Roaming
        3.1.14.  Inter-AC roaming
        3.1.15.  Intra-AC roaming
        3.1.16.  Roaming decision
        3.1.17.  Roam failure
        3.1.18.  Beacon period
        3.1.19.  Listen interval
        3.1.20.  Preauthentication
        3.1.21.  PMKID caching
        3.1.22.  QoS differentiation
    3.2.  Data plane terminology
        3.2.1.  Aggregate Multicast Throughput
        3.2.2.  Aggregate Maximum Multicast Forwarding Rate
        3.2.3.  QoS threshold
    3.3.  Control plane terminology
        3.3.1.  Endstation capacity
        3.3.2.  Endstation association rate
        3.3.3.  WTP capacity
        3.3.4.  Roaming rate
        3.3.5.  Roaming delay
        3.3.6.  Reset duration
        3.3.7.  Reset recovery time
        3.3.8.  WTP association rate
        3.3.9.  AC reset recovery time
        3.3.10.  Failover recovery time

4.  Security Considerations

5.  IANA Considerations

6.  References
    6.1.  Normative References
    6.2.  Informative References

§  Authors' Addresses
§  Intellectual Property and Copyright Statements

1.  Introduction

Wireless LANs (WLANs) are deployed on a large scale in traditional enterprises, in commercial service offerings such as coffee shops, as well as vertical applications such as inventory management. These deployments initially used a distributed network of Access Points (APs). Large WLAN deployments using distributed APs, however, introduce several issues. These include: an increased administrative burden, as the APs are individually IP-addressable; the need to ensure consistency of configuration across all APs; the difficulty of dealing with the dynamic nature of the WLAN medium and combating RF interference and noise; assuring seamless mobility for end users across the WLAN; and the more complex task of securing the WLAN against intrusion or unauthorized access.

To address the above problems, vendors offer solutions that combine aspects of LAN switching, centralized control, and distributed wireless access in an architecture comprising a set of relatively simple Wireless termination points (WTPs) coupled to one or more Access controllers (ACs). The use of centralized control and monitoring simplifies many of the management and security issues noted above, as the WTPs can be configured and controlled as a group by the ACs, security policies can be administered on a WLAN-wide basis, and the RF domain can be monitored and controlled from a central location.

Each vendor offering such a system needs a protocol between ACs and WTPs to support both centralized management and data transport functions. The general practice has been for vendors to use a proprietary protocol; however, the CAPWAP (Control and Provisioning of Wireless Access Points) protocol is being standardized by the IETF to provide a multi-vendor interoperable interface between WTPs and ACs. The CAPWAP protocol also defines a standardized WLAN architecture and mandatory functions (such as discovery) to enable a common functional model to be adopted across the vendor base.

The ACs may perform both control plane and data plane functions within a WLAN. It is therefore of significant interest to benchmark their performance, as they have a material impact on the performance and perceived end-user experience of WLANs built around them. ACs may be benchmarked either as stand-alone entities, or in conjunction with the WTPs to which they connect. When ACs are benchmarked in conjunction with WTPs, the CAPWAP architectural model is used as a reference.

This document defines the terminology to be used by vendors and users of switched WLAN devices when measuring and reporting performance characteristics of such devices. It extends existing terminology defined in RFC 2544 [RFC2544] (Bradner, S. and J. McQuaid, “Benchmarking Methodology for Network Interconnect Devices,” March 1999.) and subsequently extended in RFC 2285 [RFC2285] (Mandeville, R., “Benchmarking Terminology for LAN Switching Devices,” February 1998.). Terms generally applicable to WLAN switching systems are defined first, followed by specific terminology relating to data plane and control plane metrics.

2.  Existing definitions and requirements

RFC 1242, "Benchmarking Terminology for Network Interconnect Devices" [RFC1242] (Bradner, S., “Benchmarking terminology for network interconnection devices,” July 1991.) and RFC 2285, "Benchmarking Terminology for LAN Switching Devices" [RFC2285] (Mandeville, R., “Benchmarking Terminology for LAN Switching Devices,” February 1998.) SHOULD be reviewed in conjunction with this document. WLAN-specific terms and definitions are also provided in Clauses 3 and 4 of the IEEE 802.11 standard [802.11] (IEEE, “ANSI/IEEE Std 802.11 "Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications," ISO/IEC 8802-11:1999(E), ISBN 0-7381-1658-0,” 1999.). Commonly used terms may also be found in RFC 1983 [RFC1983] (Malkin, G., “Internet Users' Glossary,” August 1996.).

For the sake of clarity and continuity, this document adopts the general template for benchmarking terminology set out in Section 2 of RFC 1242. Definitions are organized in alphabetical order, and grouped into sections for ease of reference.

The following terms are assumed to be taken as defined in RFC 1242 [RFC1242] (Bradner, S., “Benchmarking terminology for network interconnection devices,” July 1991.): Throughput, Latency, Constant Load, Frame Loss Rate, and Overhead Behavior. In addition, the following terms are taken as defined in RFC 2285 [RFC2285] (Mandeville, R., “Benchmarking Terminology for LAN Switching Devices,” February 1998.): Forwarding Rates, Maximum Forwarding Rate, Loads, Device Under Test (DUT), and System Under Test (SUT).

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.) [RFC2119].

3.  Definitions of terms

The terminology defined in this document is divided into three main categories: general WLAN definitions, WLAN data plane definitions, and WLAN control plane definitions.

General WLAN definitions relate to the main physical and logical components of the WLAN architecture as defined by CAPWAP. These definitions are not specific to any particular metric or test procedure.

WLAN data plane definitions relate to data frame forwarding functions within the WLAN architecture.

WLAN control plane definitions are associated with functions, metrics and test procedures that are connected with the management and control of system components. These control plane functions are usually unrelated to traffic forwarding, and handled by software on the AC and WTP.

3.1.  General terms

3.1.1.  Wireless LAN (WLAN)

Definition:

A radio-based local area network architecture.

Discussion:

A WLAN performs the same functions as a typical (Ethernet) LAN, but utilizes digital radio links rather than copper or optical fiber connections. The principal benefits of a WLAN are mobility and reduced physical plant (i.e., cabling).

The most common WLAN protocol today is IEEE 802.11 [802.11] (IEEE, “ANSI/IEEE Std 802.11 "Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications," ISO/IEC 8802-11:1999(E), ISBN 0-7381-1658-0,” 1999.), which comprises: various radio PHY technologies; a Carrier Sense Multiple Access / Collision Avoidance (CSMA/CA) shared-medium access method; encryption and authentication for security; prioritized medium access for QoS; and functions for discovery, connection establishment and mobility (roaming).

Measurement Units:

N/A

Issues:

None

See Also:

Access controller (AC)

Wireless termination point (WTP)

Endstation (STA)

Service set identifier (SSID)

3.1.2.  Access controller (AC)

Definition:

A network infrastructure device that provides centralized control, management and frame switching functions for a WLAN.

Discussion:

An AC is a component of a centralized WLAN architecture as defined by CAPWAP. The AC provides connectivity and control for Wireless Termination Points (WTPs), switches frames between the wireless and wired portions of the LAN infrastructure, and supports discovery, initialization, configuration, management and data transfer functions.

Measurement Units:

N/A

See Also:

Wireless termination point (WTP)

Endstation (STA)

Wireless LAN (WLAN)

3.1.3.  Endstation (STA)

Definition:

A user or subscriber device within a WLAN, that connects to a WTP and sources or sinks data traffic.

Discussion:

Endstations in a WLAN are equivalent to hosts in a traditional (wired) LAN, but are more diverse and functional entities. They are also frequently referred to as clients. Endstations comprise normal host devices such as laptops; peripherals such as printers; mobile devices such as phones; consumer electronics devices such as TVs; and even industrial or medical devices such as RFID tags and heart monitors.

Endstations in a WLAN are connection-oriented, and must connect to the WLAN and authenticate with it before exchanging data traffic.

Measurement Units:

N/A

See Also:

Access controller (AC)

Wireless termination point (WTP)

3.1.4.  Wireless termination point (WTP)

Definition:

A bridge between the wireless (RF) and wired domains, that acts in conjunction with an AC to provide services to endstations.

Discussion:

WTPs (also referred to as Access Points, or APs) are a component of the CAPWAP architecture. They implement all of the PHY layer functions as well as some subset of the link layer functions of the IEEE 802.11 protocol. In a CAPWAP system, the WTPs serve as an encapsulating bridge between the RF domain in which the endstations exist and the wired Ethernet LAN where the ACs reside. Depending on the specific implementation, WTPs may also provide some of the discovery, security and mobility functions required by endstations.

Each WTP typically serves several endstations. The WTP and its associated endstations contend for and share the RF medium among themselves. WTPs may contain multiple radios (and MAC functionality) to provide service on more than one WLAN frequency band at the same time. WTPs may also advertise several Service set identifiers (SSIDs) on the same frequency band in order to support multiple logical WLANs that provide different types of service (e.g., voice and data) simultaneously.

Measurement Units:

N/A

See Also:

Access controller (AC)

Endstation (STA)

3.1.5.  Service set identifier (SSID)

Definition:

Tag assigned to a specific logical WLAN.

Discussion:

WLAN administrators may elect to overlay multiple logical WLANs on a single physical topology, in the same manner as virtual LANs (VLANs) are used in Ethernet LANs. Each logical WLAN is given an identifying tag, referred to as an SSID, and is a sequence of up to 32 ASCII characters. SSIDs supported by a WLAN are frequently advertised in protocol frames such as beacons and probe responses, so that endstations may discover the available list of logical WLANs and attempt to associate with one of them.

The logical WLAN identified by an SSID has a distinct set of properties shared among all its entities, such as security type (encryption as well as authentication), QoS mechanisms, access rights (e.g., guest or intranet user), and even services offered (e.g., voice or data).

Measurement Units:

N/A

Issues:

None

See Also:

Wireless LAN (WLAN)

Access controller (AC)

Wireless termination point (WTP)

Endstation (STA)

3.1.6.  Rogue device

Definition:

An unauthorized device, either an endstation or a WTP, that is introduced into a WLAN.

Discussion:

As WLANs require no physical cabling between WTPs and endstations, it is relatively simple for users or intruders to introduce a new endstation - or, less commonly, a new WTP - without the consent or knowledge of the network administrator(s). Such rogue devices are high security risks. When introduced by legitimate users, they fall outside security policies set up by the network administrators; when set up by intruders, they represent security breaches or denial of service attacks.

Detection and isolation or containment of rogue devices is a usual function in most enterprise WLANs.

Measurement Units:

N/A

See Also:

Wireless LAN (WLAN)

Access controller (AC)

Wireless termination point (WTP)

Provisioned WTP

Unprovisioned WTP

3.1.7.  Provisioned WTP

Definition:

WTPs that have been successfully registered with an AC.

Discussion:

As part of rogue WTP detection and containment procedures, a WTP is usually required to be authenticated and registered by an AC before it is allowed to associate endstations or transfer data traffic. In many cases, encrypted tunnels between the WTP and AC must be configured and firmware versions verified (or downloaded) as well. A provisioned WTP is therefore one that has successfully completed all these steps and appears in the AC database as a legitimate WTP.

Note that in some cases a WTP that has been previously authenticated and registered by an AC will take less time to be provisioned on subsequent registration attempts, because of caching of WTP information by the AC.

Measurement Units:

N/A

See Also:

Access controller (AC)

Wireless termination point (WTP)

Rogue device

Unprovisioned WTP

3.1.8.  Unprovisioned WTP

Definition:

A WTP that has not (yet) been successfully registered with an AC.

Discussion:

Unprovisioned WTPs are devices in the process of registering with an AC, but have not yet completed the registration process and are not yet treated as rogue WTPs. If an unprovisioned WTP completes the registration process successfully, it becomes a provisioned WTP; if it fails, it becomes a rogue WTP. An unprovisioned WTP is not normally capable of transferring data or associating endstations.

Measurement Units:

N/A

See Also:

Access controller (AC)

Wireless termination point (WTP)

Rogue device

Provisioned WTP

3.1.9.  Unicast traffic flow

Definition:

A stream of one or more data frames from a single source to a single intended destination.

Discussion:

For the purposes of this document, a unicast traffic flow has two attributes: it carries user or application data, and it has unicast destination addresses at both the link layer and the IP layer. (If the traffic frames are encapsulated, then these attributes pertain to the encapsulated traffic frames and not the outer headers.) A well-formed unicast traffic flow also possesses unicast source addresses at both the link and IP layers.

"User or application data" in this context refers to traffic that is not connected with control or management functions specific to the WLAN itself.

A unicast traffic flow is also expected to carry traffic for a single data stream, with reference to the protocol layer of interest. For example, if the internetwork layer is of interest, then a unicast traffic flow SHOULD carry packets from a single IP address to a single IP address. If the application layer is being considered, then the unicast traffic flow SHOULD be generated by a single application entity, and SHOULD NOT contain packets multiplexed from several application entities or protocols.

Measurement Units:

N/A

See Also:

Multicast traffic flow

3.1.10.  Multicast traffic flow

Definition:

A stream of one or more data frames from a single source to more than one intended destination.

Discussion:

For the purposes of this document, a multicast traffic flow has two attributes: it carries user or application data, and it has multicast destination addresses at both the link layer and the IP layer. (If the traffic frames are encapsulated, then these attributes pertain to the encapsulated traffic frames and not the outer headers.) A well-formed multicast traffic flow also possesses unicast source addresses at both the link and IP layers.

"User or application data" in this context refers to traffic that is not connected with control or management functions specific to the WLAN itself.

While it is possible for multicast traffic flows to be originated from different source entities, a given multicast traffic flow SHOULD be associated with a single application.

Measurement Units:

N/A

See Also:

Unicast traffic flow

3.1.11.  Best-effort traffic

Definition:

A unicast or multicast traffic flow that is not associated with any service guarantees, such as maximum loss, maximum latency, or maximum jitter.

Discussion:

This type of traffic is assigned the lowest priority in the QoS hierarchy. It is often used as the "default" traffic priority, and is typically used by applications that are not adversely affected by loss or delay. For example, a file transfer carried out over TCP can recover from network losses, and retains high transfer rates even over high-delay links.

Measurement Units:

N/A

See Also:

Unicast traffic flow

Multicast traffic flow

High-priority traffic

3.1.12.  High-priority traffic

Definition:

A unicast or multicast traffic flow that is required to adhere to one or more service guarantees, such as maximum loss, maximum latency, or maximum jitter.

Discussion:

High-priority traffic is generated and consumed by applications that cannot tolerate loss, delay (or delay variation), or both. An example is a voice connection (Voice over IP, or VoIP); voice data streams must be delivered by the infrastructure with low delay and jitter, and with low packet loss, to avoid significant degradation of voice quality.

High-priority traffic is usually marked by a QoS tag (e.g., a DSCP codepoint, an 802.11e Access Category value [802.11e] (IEEE, “IEEE Std 802.11e-2005, "Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications Amendment 8: Medium Access Control (MAC) Quality of Service Enhancements", ISBN 0-7381-4772-9,” 2005.), or an 802.1D user_priority field) that indicates the desired level of service guarantees required. In some connection-oriented networks, the marking may be implicit (i.e., assigned to the connection at setup time).

To enable the service guarantees of different types of traffic to be met, a hierarchy of QoS tags or markings is usually established within the network infrastructure. Concurrently received traffic bearing different QoS tags are provided different levels of service.

Measurement Units:

N/A

See Also:

Unicast traffic flow

Multicast traffic flow

Best-effort traffic

3.1.13.  Roaming

Definition:

The process whereby a WLAN endstation moves from the coverage area of one WTP to the coverage area of another.

Discussion:

IEEE 802.11 is a connection-oriented protocol where the endstations (clients) associate, or connect, to a single WTP at a time; typically, the closest WTP to the endstation. When an endstation moves from the neighborhood of one WTP to the neighborhood of another, it attempts to maintain the best possible wireless link parameters (signal strength and bit error ratio) by disassociating from the former and associating with the latter. At this point, the endstation is said to have roamed from one WTP to another, and a roaming event is said to have taken place.

The exact sequence of events during roaming varies considerably, based on security mode and other factors. However, the general process is as follows:

• the endstation makes a roaming decision based on some internal criteria, such as link quality or traffic load
• the endstation determines the new WTP with which it should associate or reassociate
• the endstation passively or actively disassociates with the current WTP, thereby interrupting data transfer
• the endstation performs a (re)association process with the new WTP
• data transfer resumes (i.e., the new WTP begins transferring data to the endstation)

Roaming MUST be applied only to movements of endstations within the same logical or physical wireless network; for example, the SSID MUST remain constant, and the security parameters MUST NOT change. The endstation IP address SHOULD NOT change, which usually confines roaming to a single subnet; however, if the WTPs or AC can proxy for the endstation on different subnet, then roaming can occur across subnets. Endstations that use DHCP MAY elect to renew the DHCP lease after each roaming event.

Note that "roaming" in the WLAN context corresponds to "handover" or "handoff" in the cellular wireless context.

Measurement Units:

N/A

Issues:

None

See Also:

Roaming decision

Roaming delay

Roaming rate

3.1.14.  Inter-AC roaming

Definition:

A roaming situation wherein a WLAN endstation moves between WTPs connected to different ACs.

Discussion:

An enterprise WLAN may comprise multiple ACs, each associated with a separate subset of WTPs, but acting in concert to present the appearance of a single integrated network to the endstations. In this case, it is possible for endstations to roam between WTPs belonging to different ACs, requiring the ACs to perform a handoff between themselves to maintain connectivity to the endstation.

This type of handoff is usually more complex than the simple case where the endstation roams only between WTPs associated with a single AC. It involves rapidly transferring security and connection state information from one AC to another, in addition to the usual requirements of reprovisioning WTPs and reassociating the endstation. Inter-AC roaming can therefore incur higher roaming delays.

Measurement Units:

N/A

Issues:

None

See Also:

Roaming

Roaming delay

Intra-AC roaming

3.1.15.  Intra-AC roaming

Definition:

A roaming situation wherein a WLAN endstation moves between WTPs connected to the same AC.

Discussion:

This is the simplest case of roaming in a WLAN employing WTPs and ACs: the endstations roam between different WTPs, but each endstation only roams within the subset of WTPs that are associated with a single AC. In this situation there is no need for ACs to transfer state information between themselves, and as a consequence roaming delays may be lower.

Measurement Units:

N/A

Issues:

None

See Also:

Roaming

Roaming delay

Inter-AC roaming

3.1.16.  Roaming decision

Definition:

The point in time at which a WLAN endstation decides to initiate a roaming process.

Discussion:

An endstation typically monitors the quality of the link between itself and the WTP to which it is currently associated, and will also monitor the neighboring WTPs whose signals it can receive. (Some endstations may also monitor the traffic load of their partner WTP as well.) When these factors cross predefined thresholds, and a "better" neighboring WTP is available, the endstation will make a decision to roam to the "better" WTP. This is referred to as the roaming decision, and is a basic part of the mobility process.

Once the roaming decision has been made, data traffic is usually interrupted, because the endstation will disassociate (passively or actively) from the current WTP and initiate the process of roaming to a new WTP. Data traffic in either upstream or downstream directions will not resume until the endstation has successfully roamed.

Measurement Units:

Not applicable.

Issues:

None

See Also:

Roaming

Roaming delay

Roaming rate

3.1.17.  Roam failure

Definition:

Failure to successfully complete the roaming process and restore data service.

Discussion:

The roaming (mobility) process is complex and can involve several layers of handshakes, especially when security protocols are involved. If one or more of these handshakes fails completely (i.e., the retries thereof all fail), then roaming will not complete and data service to the roaming endstation will not be restored by the WLAN infrastructure. Further, it is also possible for the association handshakes during roaming to complete successfully but for data service to not be restored (i.e., downstream data transfer does not resume). The occurrence of either of these events is classified as a roaming failure.

The consequences of a roaming failure are usually catastrophic for user applications that are engaged in transferring data while roaming is taking place.

Measurement Units:

Not applicable.

Issues:

None

See Also:

Roaming

Roaming delay

Roaming rate

3.1.18.  Beacon period

Definition:

The nominal time interval between successive beacon frames emitted by a single WTP.

Discussion:

To permit efficient discovery and self-configuration by endstations, WTPs periodically broadcast IEEE 802.11 control frames referred to as beacons. Beacons contain information relating to a logical WLAN, such as acceptable PHY data rates, security information, QoS information, and so on. The configured time interval from the start of one beacon to the start of the next beacon is referred to as the beacon period.

Note that beacons must obey the rules of the CSMA/CA access protocol as well. Hence the actual time interval between beacons may not equal the configured time interval. However, the error is not accumulative, and so over a statistically significant period of time the average interval between beacons will be very close to the nominal beacon period.

The default beacon period of IEEE 802.11 WTPs is 102.4 msec.

Measurement Units:

Time Units (one Time Unit, or TU, equals 1024 microseconds)

Issues:

None

See Also:

Listen interval

3.1.19.  Listen interval

Definition:

The time interval at which an endstation in sleep mode wakes to check for buffered traffic.

Discussion:

WLAN endstations are frequently battery-powered, mobile devices that need to conserve power consumption (and increase battery life) by sleeping when they have no data to send, after first notifying the WTP/AC. WTPs and/or ACs are expected to buffer downstream data traffic for sleeping endstations until the endstation wakes up and polls for buffered data, which is done periodically.

The interval between polls for buffered data is known as the listen interval. The size of the listen interval is usually configurable, in order to strike a balance between battery life and traffic delays. A larger listen interval results in reduced power consumption, as the endstation has to wake up less often, but also increases the delays in downstream traffic.

As the polling operation is linked to trigger fields in the beacons emitted by the WTP, the listen interval is defined to be an integral number of beacon periods. The default listen interval for normal IEEE 802.11 endstations is usually 3 beacon periods.

Measurement Units:

Beacon periods.

Issues:

None

See Also:

Beacon period

3.1.20.  Preauthentication

Definition:

The process of authenticating with a target WTP prior to performing a roam to that WTP.

Discussion:

Endstations normally need to perform a complete security handshake (e.g., IEEE 802.11 authentication and association, EAP/TLS authentication, and key exchange) when they re-associate with a new WTP during a roaming event. This is because WLAN security mechanisms produce different keys on different WTPs, as the WTP MAC address and other information forms part of the key derivation mechanism. This can greatly extend the time required to complete a roaming event, because EAP handshakes are usually long and complex.

To reduce the roaming time, the IEEE 802.11 standard permits a shortcut to be implemented by endstations: prior to the actual roam, the endstation may complete an authentication with the target WTP via the WTP with which the endstation is currently associated. Essentially the authentication frames are received over the wireless medium by the current WTP and then tunneled over the wired infrastructure to the target WTP.

When the roam event actually occurs, the endstation can bypass the entire EAP authentication process and skip directly to IEEE 802.1X key derivation, using the parameters (such as keying material) that have already been negotiated. This is known as preauthentication.

Measurement Units:

N/A

Issues:

None

See Also:

Roaming delay

PMKID caching

3.1.21.  PMKID caching

Definition:

The process of retaining (caching) security contexts when roaming between WTPs, and using the cached contexts to reduce roaming delay.

Discussion:

As described in section 3.1.14 (Preauthentication), endstations must normally perform a complete security handshake during a roaming event, which can significantly drive up roaming delay. PMKID caching is a means of reducing roaming delay in the event that an endstation returns to a WTP after roaming away from it.

Normally the endstation discards the security context when disassociating from a WTP. When PMKID caching is in effect, however, the endstation retains the security context and, if it attempts to re-associate with the same WTP, signals the WTP that the security context from the previous association is still available. The WTP may then bypass the entire EAP authentication process and skip directly to IEEE 802.1X key derivation.

Measurement Units:

N/A

Issues:

None

See Also:

Roaming delay

Preauthentication

3.1.22.  QoS differentiation

Definition:

Differential handling of traffic flows based on QoS parameters assigned to them.

Discussion:

WLANs are particularly subject to congestion issues, as not only is the capacity of a radio link comparatively low, but it is also shared by multiple endstations and WTPs. It is therefore essential to support differential handling of traffic based on their individual QoS requirements. For example, voice traffic (which is highly delay sensitive but does not occupy much bandwidth) should be permitted to take precedence over data traffic, whenever both types of traffic are contending for use of the wireless medium. This process is referred to as QoS differentiation.

QoS differentiation may be realized in many different ways. A common method is to use simple prioritization of traffic, with delay-sensitive traffic having higher priority for medium access versus best-effort traffic. The IEEE 802.11e standard [802.11e] (IEEE, “IEEE Std 802.11e-2005, "Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications Amendment 8: Medium Access Control (MAC) Quality of Service Enhancements", ISBN 0-7381-4772-9,” 2005.) specifies a traffic prioritization and differential medium access scheme suitable for IEEE 802.11 WLANs.

Measurement Units:

N/A

Issues:

None

See Also:

QoS Threshold

3.2.  Data plane terminology

The terminology in the following subsections relate directly to metrics and measurement procedures for data plane functions.

3.2.1.  Aggregate Multicast Throughput

Definition:

The maximum aggregate offered load of a set of one or more multicast traffic flows that can be presented to a DUT/SUT and forwarded without frame loss.

Discussion:

As per RFC 1242 [RFC1242] (Bradner, S., “Benchmarking terminology for network interconnection devices,” July 1991.), a throughput figure allows vendors to report a single value that has proven to be useful in the marketplace, and also correlates to the quality of the end-user experience. In some multicast protocols (e.g., push-to-talk voice), the loss of even a single multicast frame at even one destination may cause noticeable voice quality impairments. It is therefore useful to know the maximum rate at which traffic from several different multicast traffic flows can be forwarded by the DUT/SUT without frame loss.

The aggregate multicast throughput is usually obtained from an iterative set of forwarding rate measurements. It is calculated by summing, over all the multicast traffic flows in the set, the product of the offered load for each flow and the number of destinations to which that flow is to be delivered.

Measurement Units:

N-octet input frames per second

Issues:

None

See Also:

Aggregate Multicast Forwarding Rate

3.2.2.  Aggregate Maximum Multicast Forwarding Rate

Definition:

The maximum aggregate forwarding rate of a set of one or more multicast traffic flows by a DUT/SUT, irrespective of frame loss.

Discussion:

The highest forwarding rate of a DUT/SUT may not coincide with the throughput, or with the forwarding rate at maximum offered load (FRMOL - see RFC 2285 [RFC2285] (Mandeville, R., “Benchmarking Terminology for LAN Switching Devices,” February 1998.)). The maximum aggregate multicast forwarding rate figure enables the intrinsic multicast handling capacity of the DUT/SUT datapath to be assessed, and is useful in situations where some degree of loss can be tolerated (e.g., multicast video).

The aggregate maximum multicast forwarding rate is usually obtained from an iterative set of forwarding rate measurements. It is calculated by summing, over all the multicast traffic flows in the set, the number of frames per second delivered to each destination of each traffic flow.

Measurement Units:

N-octet frames per second

Issues:

None

See Also:

Aggregate Multicast Throughput

3.2.3.  QoS threshold

Definition:

A predefined set of minimum acceptable values for specific properties of a forwarded unicast or multicast traffic flow.

Discussion:

Traffic flows such as voice and video need to adhere to a certain level of service quality in order for the application layer to provide an adequate level of service to the end user. For example a voice traffic flow must keep packet losses and delays below specific levels; otherwise, the audio quality at the receiving end will be poor. A test of the QoS-related performance of a DUT/SUT must therefore also specify the minimum acceptable values for measurable parameters of the forwarded traffic flow(s), in order to determine the performance of the DUT/SUT.

For most RTP-based traffic, four parameters are of interest in this regard: the average latency, the smoothed interarrival jitter (see RFC 3550 [RFC3550] (Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, “RTP: A Transport Protocol for Real-Time Applications,” July 2003.)), the average packet loss, and the loss burst distribution. The QoS threshold for a test specifies the maximum acceptable levels of these parameters.

QoS thresholds vary depending on the needs of the application; for G.711 VoIP flows, for instance, a generally accepted QoS threshold is an average latency of 60 msec, a maximum jitter of 30 msec, a maximum packet loss of 1%, and a maximum burst loss of 1 packet. The specific QoS threshold used for a test MUST be reported.

Once a QoS threshold is defined, the test may be conducted by varying one or more test conditions - e.g., the offered load - while ensuring that the measured values of the above four parameters do not exceed the QoS threshold.

Measurement Units:

N/A

Issues:

None

See Also:

QoS differentiation

3.3.  Control plane terminology

The terminology in the following subsections relate directly to metrics and measurement procedures for control plane functions.

3.3.1.  Endstation capacity

Definition:

The maximum number of wireless endstations that can be simultaneously associated with a DUT/SUT.

Discussion:

The number of wireless endstations that can be concurrently supported by a WLAN is an important metric, as it affects the scalability of the WLAN. Further, the supported endstations cannot merely achieve the state of being associated; they must be able to transfer data as well. If data cannot be forwarded to (or from) an associated endstation by the DUT/SUT, then the endstation should not be counted towards the maximum number of concurrently supported endstations.

Measurement Units:

endstations

Issues:

None

See Also:

Endstation (STA)

Access controller

Endstation association rate

3.3.2.  Endstation association rate

Definition:

The sustained rate at which wireless endstations can successfully associate with wireless controller.

Discussion:

The rate at which wireless endstations associate with a WLAN is an important metric, as it affects both the scalability and the responsiveness of the WLAN. Further, the associated endstations cannot merely achieve the state of being associated; they must all be able to transfer data as well. If data cannot be forwarded to (or from) an associated endstation by the DUT/SUT, then the endstation should not be counted towards the association rate measurement.

Measurement Units:

endstations associated per second

Issues:

None

See Also:

Endstation (STA)

Access controller

Endstation capacity

3.3.3.  WTP capacity

Definition:

The maximum number of WTPs that can be simultaneously provisioned by an AC.

Discussion:

The number of WTPs that can be discovered and concurrently maintained by an AC (i.e., the DUT/SUT) affects the scalability of the WLAN and is therefore an essential metric. Further, the supported WTPs cannot merely achieve the state of being provisioned; they must be able to associate endstations and transfer data to/from them as well. If endstations cannot be associated, or data cannot be forwarded to (or from) associated endstations via a WTP, then it should not be counted towards the maximum number of concurrently supported WTPs.

Measurement Units:

WTPs

Issues:

None

See Also:

Access controller

Provisioned WTP

Endstation capacity

3.3.4.  Roaming rate

Definition:

The rate at which endstations can roam from the coverage area of one WTP to another (i.e., between WTPs) without failure.

Discussion:

A busy enterprise network, particularly one containing a number of VoIP handsets (i.e., mobile phones) may see a large number of roaming events per second as devices move between WTP coverage areas. In such situations it is essential to ensure that roaming takes place without failures (e.g., failed reassociations, or excessively delayed reassociations). This is particularly true in the case of VoIP handsets, where a roaming failure can cause dropped calls or degraded voice quality. The maximum roaming rate that can be sustained by the WLAN infrastructure is therefore of significant interest.

The roaming rate measurement is invalidated by the presence of roaming failures. The failure of the association handshakes with the new WTP is relatively straightforward to determine (as the handshakes themselves convey status messages and have predefined timeouts and retry limits), but the length of time to wait for resumption of data service is not well defined. A delay threshold is therefore defined as part of the measurement process, and a roam is considered to have failed if this threshold is exceeded.

The distribution of endstations among WTPs within the DUT/SUT usually has material impact on the roaming rate and therefore SHOULD be specified by the measurement procedure.

Measurement Units:

roams per second

Issues:

None

See Also:

Endstation (STA)

Wireless termination point (WTP)

Roaming

Roaming failure

Roaming delay

Endstation distribution

3.3.5.  Roaming delay

Definition:

The time interval between the initiation and successful completion of a roaming event.

Discussion:

Movement of an endstation between WTP coverage areas, which causes roaming events to occur, requires some finite time to complete, during which data traffic may be interrupted. The duration of such data traffic interruption has a material impact on the perceived capability of the WLAN to support mobile endstations. For instance, an excessive period of traffic interruption may cause voice calls to be dropped or TCP connections to close their windows. A measurement of roaming delay is thus a useful component of WLAN infrastructure performance.

The roaming delay is ideally measured from the point at which the roaming decision is made to the point at which data service is completely restored (the roam completes without a roaming failure). With reference to the WLAN infrastructure, therefore, the roaming delay is the time from the roaming decision to the first valid downstream data frame delivered to the endstation by the new WTP.

In some situations the exact point at which the roaming decision is made cannot be accurately ascertained, such as when using actual endstations rather than dedicated test equipment during the measurement process. In this case, the roaming delay is measured as the time between the last valid data frame delivered to the endstation by the old WTP to the first valid data frame delivered to the endstation by the new WTP. The use of this alternative measurement process MUST be noted along with the measurement results.

In both cases, delivery of a data frame implies that the endstation has received the data frame without error and has issued, or has been observed to issue, an IEEE 802.11 acknowledgement packet for that data frame.

A roaming event MUST NOT be counted towards roaming delay measurements if a roaming failure occurs.

The distribution of endstations among WTPs within the DUT/SUT usually has material impact on the roaming delay and therefore SHOULD be specified by the measurement procedure.

Measurement Units:

milliseconds

Issues:

None

See Also:

Endstation (STA)

Wireless termination point (WTP)

Roaming

Roaming decision

Roaming failure

Roaming rate

Endstation distribution

3.3.6.  Reset duration

Definition:

The time period for which a reset is applied to the DUT/SUT, or the time period for which the DUT/SUT is physically powered down.

Discussion:

In reset recovery and failover recovery measurements, the duration for which a reset or power-off is applied can frequently affect the test results. If the reset or power-off is applied for too short a period, anomalous behavior can occur (such as the DUT/SUT failing to reset at all). If the reset is applied for too long a period, then unwanted effects such as TCP connections being closed or association context being cleared can interfere with the test. Reset or power-off should therefore be applied to the DUT/SUT for a period sufficient to ensure a full reset of the DUT/SUT, but not long enough to cause higher-layer connections to time out.

Measurement Units:

seconds

Issues:

None

See Also:

Reset recovery time

Failover recovery time

3.3.7.  Reset recovery time

Definition:

The time period from the removal of reset from the DUT/SUT (or the power-up of the DUT/SUT) to the resumption of normal operation.

Discussion:

The reset recovery time quantifies the duration of service outage after a catastrophic event such as a power interruption. It consists of two parts: the AC reset recovery time, and the product of the rate of WTP association multiplied by the number of WTPs present. The AC recovery time forms the fixed component of the reset recovery time, while the latter product forms the variable component. As the SUT increases in scale (i.e., comprises a larger number of WTPs) the reset recovery time usually increases proportionally.

Reset recovery time = AC reset recovery time + (Rate of WTP association * Number of WTPs)

The measurement of the reset recovery time MUST include the time required for all of the WTPs to reassociate the endstations and begin transferring data. Until the last WTP has successfully restored service to the last endstation, the reset recovery process cannot be considered complete.

Measurement Units:

seconds

Issues:

None

See Also:

WTP association rate

AC reset recovery time

Failover recovery time

3.3.8.  WTP association rate

Definition:

The rate at which WTPs can be discovered and provisioned by an AC.

Discussion:

The number of WTPs present in a large enterprise WLAN may range into the thousands, with each AC being called upon to provision and manage hundreds of WTPs at a time. The rate at which the ACs can discover and provision (i.e., associate) WTPs therefore limits the speed with which the WLAN can recover from catastrophic events such as power interruptions or connectivity changes, and thus constitutes an important metric.

This metric is associated with the reset recovery test, and forms the variable portion of the reset recovery time. A WTP is not considered to be successfully provisioned by an AC until it has reassociated all the endstations formerly associated to it and begins transferring data to/from them.

Note that caching of WTP information by an AC can increase the WTP association rate for all subsequent associations after the first one. See section 3.1.7.

Measurement Units:

WTPs per second.

Issues:

None

See Also:

Provisioned WTP

Reset recovery time

AC reset recovery time

Failover recovery time

3.3.9.  AC reset recovery time

Definition:

The time period from the removal of reset from the DUT/SUT (or the power-up of the DUT/SUT) to the successful provisioning of the first WTP.

Discussion:

The AC reset recovery time is the fixed portion of the reset recovery time, and indicates the minimum time required for an AC with at least one connected WTP to recover from a catastrophic event such as a power interruption. This time is mainly required for the AC to restart and reinitialize itself, discover the first WTP, provision it, reassociate the first endstation on the WTP, and finally begin transferring data to the endstation. (If the DUT/SUT contains or connects to more than one WTP, then the actual reset recovery time is extended by the time required to provision the additional WTPs and resume normal operation.)

The measurement of the AC reset recovery time MUST include the time required for one WTP to reassociate one endstation and begin transferring data to it.

Measurement Units:

seconds

Issues:

None

See Also:

Provisioned WTP

Reset recovery time

WTP association rate

Failover recovery time

3.3.10.  Failover recovery time

Definition:

The time required to restore service after a catastrophic hardware or software event in the DUT/SUT.

Discussion:

The availability of the WLAN infrastructure is of interest when assessing the uptime and service quality of an enterprise network. In fact, to ensure uninterrupted availability, vendors often provide redundant elements in their systems (e.g., a backup AC to take over when the primary AC fails). Further, if an AC is manually reset, it should recover from the reset condition, reconnect all the attached WTPs, and restore service to endstations as rapidly as possible; this avoids issues such as dropped voice calls and failed TCP sessions.

The recovery time in the case of either a reset recovery"> measurement, or a failover recovery measurement, is measured from the onset of the reset or failure event (e.g., a forced failover signal) to the restoration of data service. Restoration of data service is deemed to have occurred when data packets are once again received by the endstations associated with the DUT/SUT.

Note that in the case of failover measurements on DUTs/SUTs incorporating hot-standby redundancy, it is possible for the recovery time to be zero, i.e., no data loss or interruption of service is noted.

Measurement Units:

seconds

Issues:

None

See Also:

Reset duration

4.  Security Considerations

Documents of this type do not directly affect the security of the Internet or of corporate networks as long as benchmarking is not performed on devices or systems connected to operating networks.

Note that performance tests SHOULD be done on with adequate isolation between the DUT and the remainder of the network, or with security systems enabled, to avoid the possibility of compromising the performance of operating networks in some manner.

5.  IANA Considerations

There are no IANA actions requested in this memo. (Note to RFC Editor: This section may be removed upon publication as a RFC.)

6.  References

6.1. Normative References

6.2. Informative References

Authors' Addresses

Full Copyright Statement

Copyright © The IETF Trust (2008).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an “AS IS” basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.