Performance Evaluation of Routing Protocol for Low Power and Lossy Networks (RPL)
draft-tripathi-roll-rpl-simulation-01

Abstract

This document presents a performance evaluation of the Routing Protocol for Low power and Lossy Networks (RPL). Detailed simulations are carried out to produce several routing performance metrics using a set of real-life scenarios.

Status of this Memo

This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on June 25, 2010.

Copyright Notice

Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License.

Table of Contents

1.  Terminology
2.  Introduction
3.  Method
4.  Simulation Setup
5.  Metrics to evaluate RPL
    5.1.  Common Assumptions
    5.2.  Path Quality
    5.3.  Routing Table Size
    5.4.  Control Packet Overhead
    5.5.  Loss of connectivity
6.  References
    6.1.  Normative References
    6.2.  Informative References
§  Authors' Addresses

1.  Terminology

PDR - Packet Delivery Ratio

Please refer to additional terminology in [I‑D.ietf‑roll‑terminology] (JP Vasseur, “Terminology in Low power And Lossy Networks, draft-ietf-roll-terminology-02 (work in progress),” May 2009.).

2.  Introduction

Designing routing in low power devices and lossy link networks imposes great challenges, mainly due to low data rates, high probability of packet delivery failure, and strict energy constraint in nodes. The IETF ROLL Working Group has specified the Routing Protocol for Low power and Lossy Networks (RPL) in [I‑D.ietf‑roll‑rpl] (Winter, T., Thubert, P., et al., “RPL: Routing Protocol for Low Power and Lossy Networks, draft-ietf-roll-rpl-04 (work in progress),” November 2009.).

RPL is designed to meet the core requirements specified in [I‑D.ietf‑roll‑home‑routing‑reqs] (Brandt, A., Buron, J., and G. Porcu, “Home Automation Routing Requirements in Low Power and Lossy Networks, draft-ietf-roll-home-routing-reqs-08 (work in progress),” September 2009.),[I‑D.ietf‑roll‑building‑routing‑reqs] (Martocci, J., Riou, N., Mil, P., and W. Vermeylen, “Building Automation Routing Requirements in Low Power and Lossy Networks, draft-ietf-roll-building-routing-reqs-07 (work in progress),” September 2009.),[I‑D.ietf‑roll‑indus‑routing‑reqs] (Pister, K., Thubert, P., Dwars, S., Phinney, T., “Industrial Routing Requirements in Low Power and Lossy Networks, draft-ietf-roll-indus-routing-reqs-06 (work in progress),” June 2009.) and [RFC5548] (Dohler, M., Watteyne, T., Winter, T., and D. Barthel, “Routing Requirements for Urban Low-Power and Lossy Networks,” May 2009.).

This document’s contribution is to provide several routing performance metrics of RPL using a decrete event simulator in various real-life deployment scenarios. Each result has been checked against several real-life deployed networks. Several routing metrics are evaluated in this document:

• Path quality metrics;
• Control plane overhead;
• Ability to cope with unstable situations (link churns, node dying);
• Required resource constraints on nodes (routing table size, etc.).

Feedback from the ROLL Working Group are welcome to add new evaluation metrics of potential interest in further revisions of this document.

Although simulation cannot prove formally that a protocol operates properly in all situations, it could give a good level of confidence in protocol behavior in highly stressful conditions, if and only if real life data are used. Simulation is particularly useful especially when theoretical model assumptions may not be applicable to such networks and scenarios. Therefore, real deployed network data traces have been used to model link behaviors.

3.  Method

RPL was simulated using OMNET++ [OMNETpp] (Varga, “The OMNeT++ Discrete Event Simulation System, in Proceedings of the European Simulation Multiconference (ESM'2001),” June 2001.), a well-known discrete event based simulator written in C++ and NED. Castalia-2.2 [Castalia‑2.2] (Boulis, A., “Castalia: Revealing pitfalls in designing distributed algorithms in WSN, in Proceedings of the 5th international conference on Embedded networked sensor systems (SenSys'07),” 2007.) has been used as Wireless Sensor Network Simulator framework within OMNET++. The output and events in the simulating are visualized with the help of the Network AniMator or NAM, which is distributed with NS (Network Simulator). (, “The Network Simulator-2, http://www.isi.edu/nsnam/ns/,” .) [NS‑2]

Note that NS or any of its versions were not used in this simulation study. Only the visualization tool was borrowed for verification purposes. As noted, real link layer data gathered from networks deployed on the field were used to compute the PDR (Packet Delivery Ratio) for each of the links in the network. By contrast with theoretical models (e.g. Markov Chains) which may have assumptions not applicable to lossy links, real-life data has been used for two aspects of the simulations:

* Link failure model: Time varying real network traces containing packet delivery probability for each link and over all channels for both indoor network deployment and outdoor network deployment were used. Thus, different types of link characteristics are used in the study.

* Topology: The topologies are gathered from real-life deployment (traces mentioned above) as opposed to random topology simulations.

4.  Simulation Setup

A 45 node topology, shown in Figure 1, gathered from a real deployment, was used in the simulations.

Figure 1

Figure 1: Network topology for preliminary simulation results.

Note that this is just a start to validate the simulation before using large scale networks.

A database of time varying link quality data, gathered from real network deployment, was created. Each link in the topology randomly 'picks up' a link model from the database, and the link’s PDR varies according to the gathered data. Packets are dropped randomly from that link with probability (1 - PDR). Each link has a PDR that varies with time (in the simulation, the new PDR is read from the database every 10 minutes). Each time a packet arrives at the Radio of a node, the module generates a random number by the Mersenne Twister Random number generation method. The random number is compared to the PDR to determine whether the packet should be dropped or not. Note that each link use a different random number generator to maintain true randomness in the simulator, and to avoid correlation between links. Also, the packet drop applies to all kinds of data and control packets (RPL) such as the DIO, DAO, DIS packets defined in [I‑D.ietf‑roll‑rpl] (Winter, T., Thubert, P., et al., “RPL: Routing Protocol for Low Power and Lossy Networks, draft-ietf-roll-rpl-04 (work in progress),” November 2009.).

In simulating RPL, the LBR first initiates sending out DIO messages, and the DAG is gradually constructed. The trickle time interval for emitting DIO message assumes the initial value of 1 second, and then changes over simulation time as mentioned in [I‑D.ietf‑roll‑rpl] (Winter, T., Thubert, P., et al., “RPL: Routing Protocol for Low Power and Lossy Networks, draft-ietf-roll-rpl-04 (work in progress),” November 2009.).

I_min is initially set to 1 second and I_doubling is equal to 16, so that maximum time between two consecutive DIO emissions by a node (under a steady network condition) is 18.2 hours. Another objective of this study is to give insight to the network administrator on how to tweak the trickle values. These recommendations could then be used in applicability statement documents. Further revision of this document will include simulations for large scale networks with varied parameters and show how quickly the network will stabilize, comparing data/control traffic and studying the tradeoff between reactivity and lifetime.

Each node in the network, other than the LBR, also emits DAO messages as specified in [I‑D.ietf‑roll‑rpl] (Winter, T., Thubert, P., et al., “RPL: Routing Protocol for Low Power and Lossy Networks, draft-ietf-roll-rpl-04 (work in progress),” November 2009.), to initially populate the routing tables with the prefixes received from children via the DAO messages in support of the Point to Point (P2P) and Point to Multipoint traffic (P2MP) in the "down" direction. In this revision of the document, it is assumed that each node is capable of storing route information for other nodes in the network. In futher revision of this document nodes without storage capability will be added to the network to see the influence of extra states on the nodes and the additional control plane overhead to propagate the route records thanks to Reverse Route Stacks in the DAO messages.

For nodes implementing RPL, the routing table memory requirement varies according to the position in the DAG. The worst-case assumption that there is no route summarization in the network is made. Thus a node closer to the DAG will have to store more routing entries. Further revision of this document will explore the influence of performing route summarization along the DAG, which could be performed thanks to a newly defined Objective Function or new address provisioning techniques. It is also assumed that all nodes have equal memory capacity to store the routing states, therefore no source routing is required.

Each node sends traffic according to a Constant Bit Rate (CBR) to all other nodes in the network over the simulation period. To simulate a more realistic scenario, 20% of the generated packets by each node are destined to the root, and the remaining 80% of the packets are uniformily assigned as destined to nodes other than the root. Therefore the root receives a considerably larger amount of data than other nodes. These values may be revised when studying the P2P traffic so as to have a majority of traffic going to all nodes as opposed to the root.

The packets are routed through the DAG built by RPL according to the mechanisms specified in [I‑D.ietf‑roll‑rpl] (Winter, T., Thubert, P., et al., “RPL: Routing Protocol for Low Power and Lossy Networks, draft-ietf-roll-rpl-04 (work in progress),” November 2009.).

Since RPL is an IP routing protocol, no assumption is made on the link layer, thus potential gains in terms of header compression provided by 6loWPAN is not under consideration [draft‑iphc] (J. Jurski, “Limited IP Header Compression over PPP, draft-jurski-pppext-iphc-02.txt (work in progress),” March 2007.).

A number of RPL parameters are configured (such as Packet Rate from each source, Time Period of the LBR emitting new DAG sequence number) to observe their effect on RPL performance metric of interest.

5.  Metrics to evaluate RPL

5.1.  Common Assumptions

Routing Table Size: as the DAO messages help to feed the routing tables in the network, routing table size for each node are recorded. Currently, the routing table size is not in terms of Kbyte of memory usage but measured in terms of number of entries for each node. Each entry has next hop node and path cost associated with the destination node. In further revision a single full 128-bit address per leaf plus a few bits to store other information and flags will be used.

The ETX (Expected Transmission Count) metric is used to build the DAG as specified in [I‑D.ietf‑roll‑routing‑metrics] (Vasseur, J., Kim, M., Networks, D., and H. Chong, “Routing Metrics used for Path Calculation in Low Power and Lossy Networks,” April 2010.). Further revisions of this document will include other metrics and constraints such as the Hop count.

5.2.  Path Quality

Number of Hops: For each pair of source and destination, the average number of hops for both RPL and shortest path routing is computed. Shortest path routing refers to an hypothetical ideal routing protocol that would always provide the shortest path in term of Total ETX (or whichever metric is used) in the network. The Cumulative Distribution Function (CDF) of hop distance for all paths (which is equal to n*(n-1) in an n node network) in the network with respect to number of hops is plotted in Figure 2 for both RPL and shortest path routing. One can observe that the CDF corresponding to 4 hops is around 80% for RPL and 90% for shortest path routing. This means, for the given topology, 90% of paths will have path length of 4 hops or less with an ideal shortest path routing methodology, whereas in RPL Point-to-Point (P2P) routing, 90% of paths will have a length shorter or equal to 5 hops. This result shows that despite having a non optimized P2P routing scheme, the path quality of RPL is not much worse than an optimized one for the topology in consideration. Another reason may be, the sink is at the center of the network, so routing through the sink is often close to an optimal (shortest path) routing. This result may be different in a topology where the sink is located at one end of the network.

Figure 2

Figure 2: CDF: hop distance versus number of hops.

Total ETX: When optimizing ETX metric along the path is used as an objective function, the total ETX along the path is computed for each pair. Figure 3 shows the CDF of the total number of ETX to deliver a packet from a source to any destination node with respect to total ETX of the path from each source to each destination for the network, for both RPL, and a shortest path mechanism. Here also one observes that total ETX along the path from all source to all destination is close to that of a shortest path for the network in our simulation.

Figure 3

Figure 3.: CDF: Total ETX along path versus ETX value.

5.3.  Routing Table Size

The objective of this metric is to observe the distribution of the the number of entries per node. Figure 4 shows the CDF of required number of routing table entries for all nodes. One can see, that 90% of the nodes need to store less than 10 entries in their routing cache.

Figure 4

Figure 4.: CDF of routing table size with respect to number of nodes.

5.4.  Control Packet Overhead

The control plane overhead is an important routing metric in Low power and Lossy Networks (LLNs). Indeed, it is imperative to bound the control plane overhead. One of the distinctive charateristics of RPL is that it makes use of trickle timers so as to reduce the number of control plane packets by eliminating redundant messages. The aim of this metric is thus to analyse the control plane overhead in stable condition (no network element failure overhead) and in the presence of failures.

Data and control plane traffic comparison for each node: Figure 5 shows the comparison of the amount of data packets transmitted (including forwarded) and control packets (DIO and DAO messages) transmitted for each node when minimizing ETX is used by the OCP along the DAG. Here one can observe that considerable amount of traffic is routed through the sink itself. And also the fact that the amount of control traffic is really negligible in the protocol is reinforced. As expected, the nodes closer to sink and that act as forwarders have much more data packet transmission than other nodes. The leaf nodes have comparable amount of data and control packet transmission, as they do not take part in routing the data.

Figure 5

Figure 5: Amount of data and control packets transmitted for each node when minimizing ETX is used OCP along the DAG.

Data and Control Packet Transmission with respect to time: In Figures 6, 7 and 8, the amount of data and control packets transmitted for node 12 (high rank in DAG), node 43 (in the middle) and node 31 (leaf node)are shown, respectively. These values stand for number of packets transmitted for each 10 minutes intervals, to help understand what is the density of data and control packet exchange in the network. One can observe as the node is closer to the sink, the amount of data is larger, and the amount of control traffic is negligible in comparison to the data traffic. Also, the variation in data traffic is much larger for a node closer to sink, because the destination of the packets varies over time, and 20% of the packets are destined to sink only. For the nodes that are further away from sink , the variation in data traffic becomes lesser, and the amount of data traffic is also smaller.

The control traffic for the nodes has a wave-like pattern. The amount of control packets for each node drops quickly as the DAG stabilizes due to the effect of trickle timer. However, as a new DAG Sequence is advertised, the trickles are reset and the nodes start emiting DIO frequently again to stabilize the DAG. One can see, for a node closer to sink, the data packet amount is much higher than control packet, and somewhat oscillatory around a mean value. The control packet amount exhibits a 'saw-tooth' behavior, mainly because as ETX was used, and as when PDR changes, ETX for a child node to its parent changes, which results in changing DAG depth of the child. This event resets the trickle timer and emit RA-DIO. Also, issue of a new DAG Sequence number triggers DAG recomputation and resets the trickle timers. Therefore, one can observe that the number of control packets attains a high value for one interval, and the amount comes down to lower values for subsequent intervals. Also, for leaf nodes the amount of control packets are more than data packets, as leaf nodes are more prone to face changes in their DAG depth as opposed to nodes closer to sink when the link PDR in the topology changes dynamically.

Figure 6

Figure 6: Amount of data and control packets transmitted for node 12.

Figure 7

Figure 7: Amount of data and control packets transmitted for node 43.

Figure 8

Figure 8: Amount of data and control packets transmitted for node 31.

5.5.  Loss of connectivity

Upon link failures, a node may loose both his parents (preferred and backup) and its sibling (if any). In this case, if a packet has to be sent and the routing table does not contain an entry for the corresponding destination the packet is dropped. RPL proposes two mechanims for DAG repairs, known as global repair and local repair. In this version of the document, simulation results are presented to evaluate the amount of time packets are lost because of loss of connectivity. The idea is to tune the frequency at which new DAGSequenceNumbers are generated by the DAG root that are used for global repair. It is expected that a higher frequency will lead to shorter duration of connectivity loss at a price of a higher rate of control packet in the network.

Figure 9 shows the CDF of time spent by any node without any service, when the packet rate from the sources is a packet each 10 seconds, and new DAG sequence number is issued every 10 minutes. This plot reflects the property of RPL without any Local Repair scheme. When all the parents (and siblings) are temporarily unreachable from a node, the time before it hears a DIO from another node is recorded, which gives the time without service. In some cases, this value might go up to the DAG Repair Timer value, because untill a DIO is heard, the link outage is not solved.

The effect of the DAG Repair Timer on time without service is plotted in Figure 10, where the source rate is 20seconds/packet and in Figure 11, where the source sends a packet every 10 seconds. In the next revision, Local Repair scheme will be implemented along with RPL to compare the service outage time with Global Repair scheme.

Figure 9

Figure 9: CDF: Loss of connectivity

Figure 10

Figure 10: CDF: Loss of connectivity for different Global Repair Period, Packet Rate 20/s

Figure 11

Figure 11: CDF: Loss of connectivity for different Global Repair Period, Packet Rate 10/s

Figure 12 shows effect of DAG Global repair timer period on control traffic. As the period to emit new DAG sequence increases, the amount of control traffic also decreases because the trickle interval gets larger for each node, which is pretty intuitive. However this smaller amount of control traffic comes at a price of increased time for loss of connectivity.

Figure 12

Figure 12: Amount of control traffic for different Global Repair Timer Period

6.  References

6.1. Normative References

6.2. Informative References

Authors' Addresses