| TOC |
|
The purpose of this document is to provide a discussion of the potential problems of using private, RFC1918, or non-globally-routable addressing within the core of an SP network. The discussion focuses on link addresses and to a small extent loopback addresses. While many of the issues are well recognised within the ISP community, there appears to be no document that collectively describes the issues.
This documents and the information contained therein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”
This Internet-Draft will expire on April 6, 2011.
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
1.
Introduction
2.
Conservation of Address Space
3.
Breaks Traceroute
4.
Breaks Path MTU Discovery
5.
Unexpected interactions with some NAT implementations
6.
Issues with edge anti-spoofing techniques
7.
Peering using loopbacks
8.
DNS Interaction
9.
Operational and Troubleshooting issues
10.
Security Arguments
11.
Issues with core network security
12.
Alternate approaches to core network security
13.
Security Considerations
14.
Normative References
Appendix A.
Acknowledgments
§
Author's Address
| TOC |
In the mid to late 90's, some Internet Service Providers (ISPs) adopted the practice of utilising private IP (i.e. RFC1918) addresses for the infrastructure links and in some cases the loopback interfaces within their networks. The reasons for this approach centered on conservation of address space (i.e. scarcity of routeable IPv4 address space), and security of the core network (also known as core hiding).
However, a number of technical and operational issues occurred as a result of using non-routable IP addresses, and as a result, virtually all these ISPs moved away from the practice. Tier 1 ISPs are considered the benchmark of the industry and today there is no known tier 1 ISP that utilises the practice of private addressing within their core network.
The following sections will discuss the various issues (and potential issues) associated with deploying private IP (i.e. RFC1918) addresses within ISP core networks.
The intent of this document is not to suggest that private IP can not be used with the core of an SP network as some providers use this practice and operate successfully. The intent is to outline the potential issues or effects of such a practice.
Note: The practice of ISPs using ‘stolen’ address space has many of the same issues (or effects) as that of using private IP address space within core networks. The term “stolen IP address space” refers to the practice of an ISP using address space for its own infrastructure/core network addressing that has been officially allocated by IANA (or an RIR) to another provider but that that provider is not currently using or advertising within the Internet. Stolen addressing is not discussed further in this document. It is simply noted as an associated issue.
| TOC |
One of the original intents for the use of private IP addressing within an ISP core was the conservation of routeable IP address space. When an ISP is allocated a block of routeable IP addresses (from IANA or an RIR), this address block was traditionally split in order to dedicate some portion for infrastructure use (i.e. for the core network), and the other portion for customer (subscriber) or other address pool use. Typically, the number of infrastructure addresses needed is relatively small in comparison to the total address count. So unless the ISP was only granted a small routable block, dedicating some portion to infrastructure links and loopback addresses (/32) is rarely a large enough issue to outweigh the problems that are potentially caused when the private address space is used as being discussed.
Additionally, specifications and equipment capability improvements now allow for the use of /31s [RFC3021] for link addresses in place of the original /30s – further minimises the impact of dedicating routeable addresses to infrastructure links by only using two (2) ip address per point to point link versus four (4) respectively.
The use of private or RFC1918 addressing as a conservation technique within an Internet Service Provider (ISP) core can cause a number of technical and operational issues or effects. The main effects are described next.
| TOC |
The single biggest effect with the use of private (RFC1918) addressing within an Internet core is the fact that it can disrupt the operation of traceroute in some situations. This section provides some examples of the issues that can occur.
Firstly, this general example below shows a traceroute across a network that uses privately numbered links. In this example, the string of " * * * " in hop numbers 19 to 21 is the result of private addresses used on those three links. The reasons for this effect are discussed following this example.
hostname% traceroute x.x.1.11 1 x.x.com (x.x.206.2) 1.081 ms 0.837 ms 0.780 ms <Part Deleted> 18 x.x.227.82 (x.x.227.82) 213.039 ms 213.407 ms 213.161 ms 19 * * * 20 * * * 21 * * * 22 x.x.1.11 (x.x.1.11) 200.192 ms !X 198.465 ms !X 198.440 ms !X
Of more relevance is the situation where the traceroute crosses an AS boundary and one of the networks has utilised private addressing. The following simple network is used for illustrative purposes.
AS100 EBGP AS200
IBGP Mesh <---------------> IBGP Mesh
Pool - Pool -
1.1.2.0/24 2.1.2.0/24
2.1.1.8/30
10.1.1.0/30 10.1.1.4/30 .9 .10 2.1.1.4/30 2.1.1.0/30
.1 .2 .5 .6 ------------ .6 .5 .2 .1
R1-----------R2-----------R3--| |--R4----------R5----------R6
10.1.1.1 10.1.1.2 10.1.1.3 2.1.1.3 2.1.1.2 2.1.1.1
Loopback Addresses shown at the bottom.
Using this example, performing the traceroute from AS200 to AS100, we can see the private addresses of the infrastructure links in AS100 are returned.
R6#traceroute 1.1.2.1 Type escape sequence to abort. Tracing the route to 1.1.2.1 1 2.1.1.2 40 msec 20 msec 32 msec 2 2.1.1.6 16 msec 20 msec 20 msec 3 2.1.1.9 20 msec 20 msec 32 msec 4 10.1.1.5 20 msec 20 msec 20 msec 5 10.1.1.1 20 msec 20 msec 20 msec R6#
While the previous example may not present a major problem, a potentially more severe case exists if the source IP address of the traceroute is within a privately numbered part of the network (AS100) and the destination is outside of the ISPs AS (AS200). In this situation the traceroute will fail completely beyond the AS boundary.
R1# traceroute 2.1.2.1 Type escape sequence to abort. Tracing the route to 2.1.2.1 1 10.1.1.2 20 msec 20 msec 20 msec 2 10.1.1.6 52 msec 24 msec 40 msec 3 * * * 4 * * * 5 * * * 6 * * * R1#
While it is completely unreasonable to expect a packet with a private source address to be successfully returned in a typical SP environment, the case is included to show the effect as it can have implications for troubleshooting. This case will be referenced in a later section.
In a complex topology, with multiple paths and exit points from the AS, the provider will loose their ability to trace paths through the network to other ASs. Such a situation could be a severe troubleshooting impediment.
It should be noted that some solutions to this problem have been proposed. They may utilise extensions to ICMP to allow the return of some "single public address" on the device, along with an interface identifier, possibly the SNMP ID of the interface. However at the time of writing, such a solution was not available.
| TOC |
The Path MTU Discovery (PMTUD) process was designed to allow hosts to make an accurate assessment of the maximum size packets that can be sent across a path without fragmentation. Path MTU Discovery is supported for TCP (and other protocols that support PMTUD such as GRE and IPsec) and works as follows:
• When a router attempts to forward an IP datagram with the Do Not Fragment (DF) bit set out a link that has a lower MTU than the size of the packet, the router MUST drop the packet and return an Internet Control Message Protocol (ICMP) 'destination unreachable - fragmentation needed and DF set (type 3, code 4)’ message to the source of the IP datagram. This message includes the MTU of that next-hop network. As a result, the source station which receives the ICMP message, will lower the send Maximum Segment Size (MSS).
It is obviously desirable that packets be sent between two communicating hosts without fragmentation as this process imposes extra load on the fragmenting router (process of fragmentation), intermediate routers (forwarding additional packets), as well as the receiving host (reassembly of the fragmented packets). Additionally, many applications, including some web servers, set the DF (do not fragment) bit causing undesirable interactions if the path MTU is insufficient. Other TCP implementations may set an MTU size of 576 bytes if PMTUD is unavailable. In addition, IPsec and other tunneling protocols will often require MTUs greater than 1500 bytes and often rely on PMTUD.
While it is uncommon these days for SP networks not to support a path MTUs in excess of 1500 bytes (with 4470 being common), the situation of 1500 byte path MTUs may still exist in some networks.
The issue is as follows:
• When an ICMP Type 3 Code 4 message is issued from an infrastructure link that uses a private (RFC1918) address, it must be routed back to the originating host. As the originating host will typically be a globally routable IP address, its source address is used as the destination address of the returned ICMP Type 3 packet. At this point there are normally no problems.
• As the returned packet will have an RFC1918 source address, problems can occur when the returned packet passes through an anti-spoofing security control (such as Unicast RPF (uRPF)), other anti-spoofing ACLs, or virtually any perimeter firewall. These devices will typically drop packets with an RFC1918 source address, breaking the successful operation of PMTUD.
As a result, the potential for application level issues may be created.
| TOC |
Private addressing is legitimately used within many enterprise or corporate networks for internal network addressing. When users on the inside of the network require Internet access, they will typically connect through a perimeter router or firewall that provides Network Address Translation (NAT) services. Typical NAT deployments assume that the internal private address ranges will not exist outside of the internal environment.
However unpredictable interactions could occur if traffic such as traceroute and PMTUD was launched from the NAT IPv4 ‘inside address’ and it passed over the same address range in the public IP core.
The discussion may be further complicated with the transition to IPv6. Current discussions around the use of NAT444 and LSN (Large Scale NAT) would make use of a double NAT process. Within this scheme, another private address block (at the time of writing) is being requested for ISP NAT 444 so that ISP private backbone space would not conflict with customer private backbone space. Again, unpredictable interactions could occur if these address ranges conflicted with the ranges assigned in an Internet core.
| TOC |
Denial of service attacks and distributed denial of attacks can make use of spoofed source IP addresses in an attempt to obfuscate the source of an attack. RFC2827 (Network Ingress Filtering) strongly recommends that providers of Internet connectivity implement filtering to prevent packets using source addresses outside of their legitimately assigned and advertised prefix ranges. Such filtering should also prevent packets with private source addresses from egressing the AS.
Best security practices for ISPs also strongly recommend that packets with illegitimate source addresses should be dropped at the AS perimeter. Illegitimate source addresses include private IP (RFC1918) addresses, addresses within the providers assigned prefix ranges, bogons (legitimate but unassigned IP addresses). Additionally, packets with private IP destinations addresses should also dropped at the AS perimeter.
If such filtering is properly deployed, then traffic either sourced from, or destined for privately addressed portions of the network should be dropped. Hence the negative consequences on traceroute, PMTUD and regular ping type traffic.
| TOC |
Although not a common technique, some ISPs use loopback addresses of border routers (ASBRs) for peering, in particular where multiple connections or exchange points exist between the two ISPs. Such a technique is used by some ISPs as the foundation of fine grained traffic engineering and load balancing through the combination of IGP metrics and multi-hop BGP (as opposed to the more common technique of local preference). When private or non-globally reachable addresses are used as loopback addresses, this technique is not possible.
| TOC |
Many ISPs utilise their DNS to perform both forward and reverse resolution for the infrastructure devices and infrastructure addresses. With a privately numbered core, the ISP itself will still have the capability to perform name resolution of their own infrastructure. However others outside of the autonomous system will not have this capability. At best, they will get a number of unidentified RFC1918 IP addresses returned from a traceroute.
Such a situation can lead to complaints from parties or customers outside of the ISP.
| TOC |
Previous sections of the document have noted issues relating to network operations and troubleshooting. In particular when private IP addressing within an ISP core is used, the ability to easily troubleshoot across the AS boundary may be limited. In some cases this may be a serious troubleshooting impediment. In other cases, it may be solved through the use of alternative troubleshooting techniques.
The key point is that the flexibility of initiating an out bound ping or traceroute from a privately numbered section of the network is lost. In a complex topology, with multiple paths and exit points from the AS, the provider may be restricted in their ability to trace paths through the network to other ASs. Such a situation could be a severe troubleshooting impediment.
For users outside of the AS, the loss of the ability to use a traceroute for troubleshooting is very often a serious issue. As soon as many of these people see a row of "* * *" in a traceroute they often incorrectly assume that a large part of the network is down or inaccessible (e.g. behind a firewall). Operational experience in many large providers has shown that significant confusion results. This translates directly into difficult (and potentially long and emotional) conversations with these users.
| TOC |
One of the arguments often put forward for the use of private addressing within an ISP is an improvement in the network security. It has been argued that if private addressing is used within the core, the network infrastructure becomes unreachable from outside the providers autonomous system, hence protecting the infrastructure. There is an element of legitimacy to this argument. Certainly if the core is privately numbered and unreachable, it potentially provides a level of isolation in addition to what can be achieved with other techniques such as infrastructure ACLs on their own. This is especially true in the event of an ACL misconfiguration, something that does commonly occur as the result of human error.
| TOC |
There are a number of flaws to the argument of using private addressing for security.
• The approach does not protect against reflection attacks. For example, if a packet with spoofed source address corresponding to the networks infrastructure address range, is sent to a host (or other device) attached to the network, that host will send its response directly to the infrastructure address. If such an attack was performed across a large number of hosts, then a successful large scale denial of service attack on the infrastructure could be achieved. This is not to say that a publicly numbered core will protect from the same attack, it won’t. The key point is that a reflection attack does get around the apparent security offered in a privately addressed core.
Even if anti-spoofing is deployed at the AS boundary, the border routers will potentially carry routing information for the privately addressed network infrastructure. This can mean that packets with spoofed addresses, corresponding to the private infrastructure addressing, may be considered legitimate by edge anti-spoofing techniques such as Unicast Reverse Path Forwarding – Loose Mode, and forwarded. To avoid this situation, an edge anti-spoofing algorithm such as Unicast Reverse Path Forwarding – Strict Mode, would be required. However, such an approach can be problematic in an environment where asymmetric traffic paths exist. Alternatively, access control lists to block packets with private or RFC1918 source addresses could be used.
• Additionally, in cases where distributed denial of service attacks make use of spoofed source addresses large amounts of backscatter traffic can be produced. Backscatter traffic is returned from hosts, etc when the received packet contained a spoofed source address. That traffic, or an ICMP destination unreachable messages are sent to an incorrect party as it was assumed to be the legitimate source address, which it wasn’t.
If the core network is privately numbered, then that backscatter traffic may be received by infrastructure devices. As backscatter traffic is very common on the Internet, the amount of backscatter traffic received by the control and management planes of the network infrastructure devices could be significant. Figures in the order of hundreds of MBits/s have occurred in some networks. Without suitable protection, the network infrastructure devices could be negatively impacted in such a situation, in particular if the spoofed address range corresponded with the private address range used to number the core. Again, a publicly numbered core does not protect from this issue. …but is likely to be less of a problem due to the wider address sizes.???
| TOC |
Today, hardware-based ACLs, which have minimal to no performance impact, are now widespread. Applying an ACL at the AS perimeter to prevent access to the network core may be a far simpler approach and provide comparable protection to using private addressing, Such a technique is known as an infrastructure ACL (iACL).
In concept, iACLs provide filtering at the edge network which allows traffic to cross the network core, but not to terminate on infrastructure addresses within the core. Proper iACL deployment will normally allow required network management traffic to be passed, such that traceroutes and PMTUD can still operate successfully. For an iACL deployment to be practical, the core network needs to have been addressed with a relatively small number of contiguous address blocks. For this reason, the technique may or may not be practical.
The other approach to preventing external access to the core is IS-IS core hiding. This technique makes use of a fundamental property of the IS-IS protocol which allows link addresses to be removed from the routing table while still allowing loopback addresses to be resolved as next hops for BGP. The technique prevents parties outside the AS from being able to route to infrastructure addresses, while still allowing traceroutes to operate successfully. IS-IS core hiding does not have the same practical requirement for the core to be addressed from a small number of contiguous address blocks as with iACLs.
These techniques may not be suitable for every network, however, there are many circumstances where they can be used successfully without the associated issues of privately addressing the core.
MPLS VPN - Yet to be written.
| TOC |
None
| TOC |
| [NANOG] | Various, “Various Nanog mail archives.” |
| [RFC1918] | Rekhter , Y., Moskowitz, R., Karrenberg, D., Jan de Groot, G., and E. Lear , “RFC1918 Address Allocation for Private Internets, BCP 5,” Febuary 1996. |
| [RFC2728] | Ferguson, P. and D. Senie , “RFC 2827 Network Ingress Filtering, BCP 38,” May 2000. |
| TOC |
The author would like to thank the following people for their input and review – Dan Wing (Cisco Systems), Roland Dobbins (Arbor Networks), Philip Smith (Cisco Systems), Barry Greene (Juniper Networks), Anton Ivanov (kot-begemot.co.uk), Ryan Mcdowell (Cisco Systems), Russ White (Cisco Systems), Gregg Schudel (Cisco Systems), Michael Behringer (Cisco Systems).
| TOC |
| Anthony Kirkham | |
| Cisco Systems | |
| Level 12 | |
| 300 Adeliade St | |
| Brisbane, Queensland 4000 | |
| Australia | |
| Phone: | +61 7 32388203 |
| Email: | tkirkham@cisco.com |