UDP and TCP as the New Waist of the Internet Hourglass
draft-rosenberg-internet-waist-hourglass-00

Status of this Memo

By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on August 15, 2008.

Abstract

One of the fundamental design principles of the Internet is that IP represents a common intermediate protocol layer, linking together a variety of link layer technologies underneath with a large number of applications on top. When drawn graphically, this can be show as an hourglass with IP in the middle. The preponderence of NATs and firewalls in the Internet has changed this reality, such that UDP and TCP are now the waist of the hourglass. This document discusses this change and describes its implications for protocol and application design.

Table of Contents

1.  Terminology
2.  Introduction
3.  What Caused the New Model?
4.  New Transport Protocols
5.  What about IPv6?
6.  Security Considerations
7.  Acknowledgements
8.  References
    8.1.  Normative References
    8.2.  Informative References
§  Author's Address
§  Intellectual Property and Copyright Statements

1.  Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.) [RFC2119].

2.  Introduction

One of the fundamental design principles of the Internet is that IP represents a common intermediate protocol layer. This intermediate layer is designed to support a variety of link layers - ethernet, fiber, ATM, and so on. Furthermore, new link layer technologies can be added in the future, without affecting IP itself.

In addition, IP can support a broad set of applications on top of it, ranging from email to instant messaging to voice over IP. All of those applications do not require changes to IP itself. As with link layers, new applications can be added at any time.

This design principle is often described as, "Everything over IP, and IP over everything". Graphically, it is often shown as an hourglass, with IP in the middle:

            +------+------+------+------+------+
            |Email | Web  | VoIP | P2P  | RTSP |
            +------+------+------+------+------+
                   | TCP  |  UDP | ICMP |
                   +------+------+------+
                          |  IP  |
                   +------+------+------+
                   |Ether |Sonet | ATM  |
            +------+------+------+------+------+
            |Fiber | TP   | CAT5 | WiFi | GSM  |
            +------+------+------+------+------+

This simple design is at the core of the success of the Internet, and is arguably the foundational principle on which it exists.

Unfortunately, there is a new reality in the Internet. Like it or not, this model is no longer true.

The preponderence of NATs and firewalls in the Internet has created a new model. In this new model, the waist of the hourglass is now UDP and TCP, with ICMP being squeezed out. The new model looks like this:

            +------+------+------+------+------+
            |Email | Web  | VoIP | P2P  | RTSP |
            +------+------+------+------+------+
                      |TCP/IP|UDP/IP|
                   +--+------+------+---+
                   |Ether |Sonet | ATM  |
            +------+------+------+------+------+
            |Fiber | TP   | CAT5 | WiFi | GSM  |
            +------+------+------+------+------+

This draft explains why this has happened and why it is now impossible to define new protocols natively on top of IP, discusses the implication for protocol design, and then considers the implications for IPv6.

3.  What Caused the New Model?

Originally, NAT was designed to operate strictly at the IP layer - translating internal to external addresses 1-for-1. This was done primarily to avoid network renumbering when there was a change in provider. However, NAT-P - NAT with port translation - quickly emerged. In NAT-P, a large number of hosts can utilize a single external IP address by using the UDP and TCP port numbers as a demultiplexing point.

In essence, the UDP or TCP port number became an extended version of the IP address - adding another 16 bits to the total amount of space, providing for 48 overall.

NAT-P in particular, often just called NAT, has seem extremely widespread deployment. Almost every residence with broadband Internet is running NAT (NAT-P to be specific). NAT is common in enterprises, and there are more and more cases of multilayer NAT within a single enterprise. Service providers have been known to NAT entire networks; for example some wireless networks give mobile devices net-10 addresses.

When put together, the implication is that basic packet transport between point A and point B is, these days, frequently possible only if the transport is UDP or TCP. This is because NAT-P requires one of these two in order to utilize the 16 bit ports as as a demultiplexing point. NATs, which are part of the network, have become not just TCP and UDP-aware, they have become TCP and UDP *dependent*. Almost every NAT in deployment today will simply discard a packet above IP that is not TCP or UDP. The primary exception is ICMP. Some NATs will pass ICMP packets, but it is filtered by many. Consequently, it only sometimes works on the Internet. This is also making it difficult to rely on. Indeed, it's success rate is sufficiently poor that new mechanisms for path MTU discovery have been designed which work without ICMP [RFC4821] (Mathis, M. and J. Heffner, “Packetization Layer Path MTU Discovery,” March 2007.).

This means that the operation of the public Internet is dependent on the existence of UDP and TCP traffic. While it is true that, in some cases you can get other transport protocols to run between two hosts, if you want RELIABLE transport on the Internet - transport that works between ANY two points - you have but two choices - UDP or TCP. Nothing else works reliably, or even close to reliably.

Its not just NAT. Firewalls are also TCP and UDP aware, and are often configured to discard non-UDP or non-TCP traffic.

4.  New Transport Protocols

Does this mean that it is impossible to define new transport protocols? Fortunately, the answer is no.

New transport protocols can be defined. However, if the intention is that these protocols ever actually work on the public Internet, they need to run on top of the only available packet transport on the Internet - UDP/IP. UDP has replaced IP as the 'raw' point to point packet transport on the Internet. If congestion control or signaling or other features are desired, they must be layered on top of UDP, rather than in a new protocol beside it.

It is tempting to design a protocol on top of IP in such a way that it is "NAT friendly". In this context, "NAT friendly" means that the protocol has been designed to make ALGs for that protocol easy to implement. However, there is a vicious cycle that prevents such ALG functionality from ever being built. New features get added to products, such as NATs, when the market demands them. The market demands them when there is enough usage to create such a demand. However, if the protocol will fail utterly to work in the face of existing NAT, there will never be enough usage to create such demand. Even if there was, the amount of time it would take to upgrade all of the NATs on the Internet to support this ALG functionality is astoundingly large. Thus, the protocol will continue to be unreliable on the Internet, since there is always a chance that it hits a NAT which doesn't support the necessary ALG functionality.

Consequently, designing protocols to be "NAT friendly" in this way does not work in practice. New applications and protocols MUST be designed to run on top of either UDP or TCP. Full stop. Examples of where this has been done after the fact is [I‑D.tuexen‑sctp‑udp‑encaps] (Tuexen, M. and R. Stewart, “UDP Encapsulation of SCTP Packets,” December 2009.) for SCTP and [RFC3948] (Huttunen, A., Swander, B., Volpe, V., DiBurro, L., and M. Stenberg, “UDP Encapsulation of IPsec ESP Packets,” January 2005.) for IPSec. However, it is far better to define this at the beginning, and furthermore, have it as the one and only mode of operation. This avoid protocol choices, and therefore simplifies design and improves interoperability.

5.  What about IPv6?

Will the IPv6 Internet share the same fate as the IPv4 Internet, and work only with UDP and TCP? It seems likely that the answer will be yes.

The primary reason is that the IPv6 Internet is not something that will appear overnight to replace the IPv4 Internet. It will run alongside it for a very long time. Hosts that have connection to the IPv6 Internet will find themselves frequently using IPv4 (in a dual-stack deployment), because the target host is available only on IPv4, or will find themselves communicating with via IPv6 to a v4-only host through NAT. In both cases, any protocols except for UDP and TCP based protocols will not work. And thus, the v6 host will need to utilize protocols that do work in all cases - ones based on UDP and TCP - rather than ones that work only in a few cases. And so, when we eventually do cutover to IPv6 only, it will be with hosts which have, all along, only been running protocols that run on top of UDP or TCP.

Another reason is that the IPv6 Internet will certainly be filled with firewalls, and if history is any guide for the future, only TCP and UDP are likely to work through such firewalls.

Finally, the IPv6 Internet may be filled with NAT anyway, despite attempts to provide ample address space.

Consequently, for an application designers perspective, why build an application on top of a protocol which doesn't work on the IPv4 Internet, and won't work on anything but a pure IPv6 network, when an application running on top of UDP or TCP will work everywhere?

6.  Security Considerations

This document does not introduce any additional security considerations for the Internet.

7.  Acknowledgements

The author would like to thank Dan Wing, Cullen Jennings, Scott Brim, Paul Kyzivat, and Dave Oran for their comments.

8.  References

8.1. Normative References

8.2. Informative References

Author's Address

Full Copyright Statement

Copyright © The IETF Trust (2008).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an “AS IS” basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.