Reference Model for IPFIX Mediators
draft-kobayashi-ipfix-mediator-model-01.txt

Status of this Memo

By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on May 22, 2008.

Abstract

An IPFIX Mediator is an intermediate device between IPFIX Exporting Processes and IPFIX Collecting Processes. IPFIX Mediators act as an IPFIX Proxy, IPFIX Firewall, and IPFIX concentrator. IPFIX Mediators mediate IPFIX protocol using several functions. That enables the flow-based measurement system to become a high-capacity system and accommodate a variety of monitoring methods. This document describes each function that is provided by IPFIX Mediators and the method of handling the Flow Records of each function. In addition, this document describes a model of an applicable scenario using IPFIX Mediators.

Table of Contents

1.  Introduction
2.  Terminology
3.  Framework for IPFIX Mediators
    3.1.  Internal Components Model
        3.1.1.  Collecting Process
        3.1.2.  Metering Process
        3.1.3.  Exporting Process
        3.1.4.  Storing Process
    3.2.  IPFIX Protocol Considerations
        3.2.1.  Export Time Issue
        3.2.2.  Observation Domain ID Management
        3.2.3.  Template Management
        3.2.4.  Transport Session Management
        3.2.5.  Option Template Management
        3.2.6.  Reporting of Exporter Information
4.  Solution Scenarios with IPFIX Mediators
    4.1.  Flexible Aggregation
    4.2.  Distributed Aggregation
    4.3.  Duplication of Flow Records
    4.4.  Distribution of Flow Records
    4.5.  Extraction of Suspicious Flow
5.  Mediator Option Template Presentation
    5.1.  Exporter Information Option Template
    5.2.  Usage of Scope Field
6.  Security Considerations
7.  IANA Considerations
8.  References
    8.1.  Normative References
    8.2.  Informative References
§  Authors' Addresses
§  Intellectual Property and Copyright Statements

1.  Introduction

Several problems regarding flow-based measurement systems have occurred in several networks, as follows. In particular, problems in large-scale networks and integrated networks are described in detail in [I‑D.kobayashi‑ipfix‑large‑ps] (Kobayashi, A., Nishida, H., Sommer, C., Dressler, F., and E. Stephan, “Problems with Flow Collection in Large-Scale Networks,” October 2007.).

• Scalability of Collector in large-scale networks
  
  As sizes of networks become larger, the number of Flow Records becomes greater. Large numbers of Flow Records have been burdening management networks and the Collecting Process. Maintaining scalability is difficult as a network grows. Generally, network operators need to monitor overall wide-scale traffic behavior and investigate detailed traffic information when traffic incidents happen. Meeting these requirements seems to be difficult for a single Collector device because of large numbers of Flow Records.
• Flow-based measurement in integrated networks
  
  On the other hand, networks such as IPv4, IPv6, and VPN on MPLS have recently become more integrated. These sorts of traffic need to be analyzed separately from a different perspective. However, handling them separately without improving the capability of the Collector is difficult.
• Exchanging traffic information between different networks
  
  Exchanging traffic information is considered to be necessary for network providers as well as each customer. To that end, the network provider needs to export specified Flow Records while avoiding privacy violations. In addition, exchanging traffic information between associated networks could be necessary when traffic incidents happen. In that case, Flow Records should be checked according to the network operator's policy before exporting. Therefore, Flow Records exported from an Exporter directly without modification can not be fed to each customer or associated network operators without modification.

IPFIX Mediators enable network operators to overcome these problems by preprocessing Flow Records. By defining IPFIX Mediators, network operators can take increasing advantage of an extensive template format, and handle Flow Records in accordance with their preference. This document describes a model of applicable scenarios by using IPFIX Mediator and its key component.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL","SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.).

2.  Terminology

The definitions of basic IPFIX and PSAMP terms are identical with those in [I‑D.ietf‑psamp‑framework] (Duffield, N., “A Framework for Packet Selection and Reporting,” June 2007.), [RFC3917] (Quittek, J., Zseby, T., Claise, B., and S. Zander, “Requirements for IP Flow Information Export(IPFIX),” October 2004.), [I‑D.ietf‑ipfix‑protocol] (Claise, B., “IPFIX Protocol Specification,” September 2007.), [I‑D.ietf‑ipfix‑info] (Quittek, J., Bryant, S., Claise, B., and J. Meyer, “Information Model for IP Flow Information Export,” February 2007.), and [I‑D.ietf‑ipfix‑architecture] (Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, “Architecture for IP Flow Information Export,” September 2006.). Other than the above terminology, the following terminology related to IPFIX Mediator is used in this document. Therefore, terms defined in the IPFIX terminology are capitalized in this document.

IPFIX Mediator

An IPFIX Mediator hosts at least one Exporting Process and one Collecting Process. IPFIX Mediator is a generic name of several devices, such as an IPFIX Proxy, an IPFIX Firewall, an IPFIX Distributor, and an IPFIX concentrator.

Original Exporter

An Original Exporter hosts Observation Points where IP packets can be observed.

IPFIX Proxy

An IPFIX Proxy acts as a proxy for Original Exporters, which host Observation Points. An IPFIX Proxy may receive Flow Records from one or several Exporting Processes, and send them to one or several Collecting Processes. An IPFIX Proxy does not send information about an Original Exporter, such as "exporterIPv{4|6}Address", to the Collector to act as an Original Exporter.

IPFIX Distributor

An IPFIX Distributor replicates Flow Records and forwards them to multiple Collectors. In addition, an IPFIX Distributor classifies Flow Records and forwards the classified Flow Records to dedicated Collectors. An IPFIX Distributor does not change the value or template of Data Records.

IPFIX Firewall

An IPFIX Firewall exports Flow Records to a different network domain at the edge of a network domain. From a Collector's point of view, an IPFIX Firewall acts as an Original Exporter, just like an IPFIX Proxy. In addition, an IPFIX Firewall reviews whether received Flow Records are passed forward to a Collector to hide the network topology or privacy information. An IPFIX Firewall has at least one function of filtering, modifying, and anonymizing functions.

Metering Process

The Metering Process in IPFIX Mediators can be considered as a partial Metering Process separated from the Metering Process in the Original Exporter. The Metering Process in IPFIX Mediators consists of a set of subprocesses that include the Selecting Process, Aggregating Process, and Modifying Process. The Metering Process generates the final Flow Records that should be exported.

Selecting Process

The Selecting Process in an IPFIX Mediator is similar to that of PSAMP Devices, which is described in [I‑D.ietf‑psamp‑framework] (Duffield, N., “A Framework for Packet Selection and Reporting,” June 2007.). However, the Selecting Process in an IPFIX Mediator only has field-match filtering functions. This filtering function blocks Flow Records based on the values of specified Information Elements to forward them to the next process. The Selecting Process is one of the subprocesses in the Metering Process.

Aggregating Process

The Aggregating Process creates aggregated Flow Records from input Flow Records in accordance with aggregation rules that are described in [I‑D.dressler‑ipfix‑aggregation] (Dressler, F., Sommer, C., and G. Munz, “IPFIX Aggregation,” June 2006.). The Aggregating Process is one of the subprocesses in the Metering Process.

Modifying Process

The Modifying Process carries out two different kinds of modification, as follows.

The Modifying Process changes the Template and record structure by adding/deleting specific Information Elements. For example, the Modifying Process adds Information Elements like derived packet properties that cannot be extracted in the Original Exporters. Information Elements related to derived packet properties are described in [I‑D.ietf‑ipfix‑info] (Quittek, J., Bryant, S., Claise, B., and J. Meyer, “Information Model for IP Flow Information Export,” February 2007.).

The Modifying Process changes the value of the specific Information Elements. For example, the values of specific Information Elements are anonymized to avoid violating privacy.

The Modifying Process is a key part of a IPFIX Firewall. The Modifying Process is one of the subprocesses in the Metering Process.

Storing Process

The Storing Process stores the input Flow Records from any process in a storage system such as a database or flat-file system. Stored data may be retrieved from the Storing Process in different ways, which are outside the scope of this document.

Distributing Function

The Distributing Function classifies input Flow Records based on the value of specified Information Elements. The classified Flow Records are exported to the specified Collectors. The Distributing Function is carried out by the Exporting Process.

Observation Domain ID

An IPFIX Mediator does not host the Observation Points and Observation Domain. The Observation Domain ID in the IPFIX header sent by IPFIX Mediator also indicates the largest set of Observation Points in the Original Exporter, but this value does not indicate the physical entity of the Original Exporter. If an IPFIX Mediator handles one Collecting session and one Exporting session, the IPFIX Mediator does not need to change the value of the Observation Domain ID. If an IPFIX Mediator handles multiple sessions on the collecting and exporting side, the IPFIX Mediator needs to assign a new value.

Transport Session Information

In SCTP, the Transport Session Information is the SCTP association. In TCP and UDP, the Transport Session Information corresponds to a 5-tuple {exporter IP address, collector IP address, exporter transport port, collector transport port, transport protocol}. In IPFIX Mediator, the Collecting Process manages this information.

3.  Framework for IPFIX Mediators

An IPFIX Mediator is located between one or more Exporting Processes and one or more Collecting Processes. An IPFIX Mediator acts as a Collector by receiving Flow Records, and it acts as an Exporter by sending Flow Records. This dual-role architecture enables cascading IPFIX Mediators and building a combination of several solutions.

The internal model of an IPFIX Mediator is composed of Collecting Processes, Metering Processes, Storing Processes, and Exporting Processes. An IPFIX Mediator may optionally have Metering Processes and Storing Processes. Basically, the allocation of these Processes depends on the role of devices, such as IPFIX Proxy, IPFIX Firewall, IPFIX Distributor, and IPFIX concentrator. "IPFIX Mediator" is a generic term for these devices. Basically, IPFIX Mediators have two types of mediation functions, as follows.

IPFIX Protocol Mediation
An IPFIX Mediator forwards input Flow Records to upstream Collectors. An IPFIX Mediator does not change the set of Flow Records nor the value or Template of Flow Records. This function applies to an IPFIX Proxy and IPFIX Distributor.

Flow Mediation
An IPFIX Mediator creates a new set of Flow Records from input Flow records. A Flow Mediation indicates Flow aggregation, selection, and modification. A Flow modification indicates two different kinds of modifications. One is changing the value of specified Information Elements. The second is changing the Template and record structure by adding/deleting specified Information Elements. This function applies to IPFIX Firewall and IPFIX concentrator.

Surely, the composite of these functions can be considered in many ways as one of the devices of IPFIX Mediators.

3.1.  Internal Components Model

The following figure indicates four components (Collecting Process, Metering Process, Exporting Process, and Storing Process) within the IPFIX Mediator that are referred to in [RFC3917] (Quittek, J., Zseby, T., Claise, B., and S. Zander, “Requirements for IP Flow Information Export(IPFIX),” October 2004.). The Metering Process can have one or multiple subprocesses. These subprocesses are the Selecting Process, Aggregating Process, and Modifying Process that can be connected to each other in any sequence defined by the user. The Metering Process and Storing Process are options.

  +--------------------------------------------------------------+
  |                        IPFIX Mediator                        |
  | .----------.                                     .---------. |
  | |          |  .-------------------------------.  |         | |
  | |Collecting|  |        Metering Process       |  |Exporting| |
  | |Process   |  |.-------.  .-------.  .-------.|  |Process  | |
  | |          |--->sub    |-->sub    |->|sub    |-->|         | |
IPFIX          |  ||process|  |process|  |process||  |         IPFIX
--> |          |.->|#1     |  |#2     |  |#3     ||  |         |-->
  | |          || |'-------'  '-------'  '-------'|  |         | |
  | |          || '----|----------|----------|----'  |         | |
  | |          ||      |          |          |       |         | |
  | |          ||   .--\/---------\/---------\/--.   |         | |
  | |          |'---|       Storing Process      |<--|         | |
  | |          |--->|                            |-->|         | |
  | |          |    '----------------------------'   |         | |
  | |          |------------------------------------>|         | |
  | '----------'                                     '---------' |
  +--------------------------------------------------------------+


Figure A: Key components within IPFIX Mediator.

Each process is associated with a common identifier in the IPFIX Mediator. This method is similar to PSAMP associations in [I‑D.ietf‑psamp‑sample‑tech] (Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. Raspall, “Sampling and Filtering Techniques for IP Packet Selection,” June 2007.).

3.1.1.  Collecting Process

The Collecting Process receives Flow Records from the previous Exporter. An instance of the process is created according to the IPFIX session. Functions of the process are described in [I‑D.ietf‑ipfix‑protocol] (Claise, B., “IPFIX Protocol Specification,” September 2007.). The process also forwards received Flow Records with IPFIX header information and Transport Session Information to multiple Metering Processes or Storing Processes. In other words, Flow Records can be duplicated by forwarding multiple Metering Processes or Exporting Processes. In addition, the process can directly forward Flow Records to the Exporting Process.

3.1.2.  Metering Process

The Metering Process generates a new set of Flow Records from input Flow Records with received IPFIX header information, such as "Export Time" and "Observation Domain ID". The process hosts several subprocesses. The processing order of these functions, which could be configured by user definitions would create a different set of Flow Records.

3.1.2.1.  Selecting Process

The Selecting Process decides whether a Flow Record passes through to the next process. The process has a filtering function and selects Flow Records that are matched under given conditions. Prior to receiving Flow Records, the user configures a filter pattern in Selecting Process, which specifies how the Flow Records are treated by the process. If the values of some Information Elements in the Flow Record match the filter pattern, this process selects Flow Records with all fields and forwards these Flow Records to the next process. For example, the process selects Flow Records that are included in the specified destination IP address.

3.1.2.2.  Aggregating Process

The Aggregating Process gathers Flow Records within a given time interval and then distinguishes Flow Records that have common properties. If values of a given key field are the same, that means those Flow Records have common properties. This process merges Flow Records that have a common property and creates an aggregated Flow Record. Therefore, for example, aggregated Flow Records have an aggregation counter that indicates the number of packets. These functions are defined in accordance with the IPFIX aggregation rule in [I‑D.dressler‑ipfix‑aggregation] (Dressler, F., Sommer, C., and G. Munz, “IPFIX Aggregation,” June 2006.).

The process has aggregation rule defined by the user prior to receiving Flow Records. The process indicates Information Elements that should become aggregated flow keys and other Information Elements that should be kept or discarded. In addition, these instruction rules include Information Elements that should be added to aggregated Flow Records. Aggregated Flow Records may need to complement information that is discarded during the Aggregating Process. They help the Collector to analyze aggregated Flow Records. For example, these Information Elements correspond to "averageActiveTime", "synCount", and "flowCount" elements, as follows.

• averageActiveTime
  
  This Information Element indicates the average active time of an input Flow Record in the aggregated Flow Records. This Information Element is created from flow time stamp Information Elements. There are "flowStartSeconds", "flowEndSeconds", "flowStartMilliSeconds", "flowEndMilliSeconds", "flowStartSysUpTime", and "flowEndSysUpTime". Moreover, "minimumActiveTime" and "maxmumActiveTime" might be considered in addition to the element.
• synCount
  
  This Information Element indicates the number of input Flow Records that have "tcpControlBits", which the SYN bit sets to 1 in an aggregated Flow Record. Using this element, Collector can determine the number of SYN packets throughout the network. Moreover, "ackCount", "finCount", "pshCount", "urgCount", and "rstCount" might be considered in addition to this element.
• flowCount
  
  This Information Element is the number of input Flow Records included in an aggregated Flow Record.

3.1.2.3.  Modifying Process

The Modifying Process modifies the input Flow Records. The process can add new Information Elements, delete included Information Elements, or modify the value of included Information Elements, as follows. If this process modifies the original template, it SHOULD revise the received "flowKeyIndicator".

Adding new Information Elements

This function adds specified Information Elements into the input Flow Records. The values of Information Elements are extracted by searching some database based on the input value of other specified Information Elements. The added Information Elements and used Information Elements are configured according to instructions by the user to obtain the value. The method to obtain a value from some Information Elements is outside the scope of this document.

This function, instead of the Original Exporter, adds a derived packet property parameter, which is useful for the traffic-monitoring technique. Doing that can compensate for the inability of some Exporters to add a derived packet property parameter. Therefore, the Collector does not need to recognize the difference between implementations of routers from several vendors. For example, the addition of "bgpNextHop{IPv4|IPv6}Address" and "bgpCommunity" Information Elements is useful for making a traffic matrix that covers the whole network domain. "bgpNextHop{IPv4|IPv6}Address" can indicate the egress router of some network domain. In addition, "bgpCommunity" can indicate the same group of destination or source IP addresses. This value can be given by looking for the BGP route database based on the destination or source IP address. In addition, "mplsVpnRouteDistinguisher", which cannot be extracted from the core router in MPLS networks, indicates the customer's identification. Network Operators can monitor the traffic behavior of each customer by adding "mplsVpnRouteDistinguisher" to the Flow Records. This value can be given by looking for the BGP route database based on the "mplsTopLabelStackSection" and "mplsTopLabel{IPv4|IPv6}Address".

Deleting Information Elements

This function deletes specified Information Elements according to instructions that are configured by the user, which indicate whether an Information Element should be removed. Hiding network topology information and private information by using this function is possible.

In the case of exchanging Flow Records with different network domains or customers, this function can avoid making a vulnerability by deleting unnecessary Information Elements. By deleting unnecessary Information Elements, this function can hide the network topology and another customer's information. In particular, "ipNextHopIP{v4|v6}Address", "bgpNextHopIP{v4|v6}Address", and "bgp{Next|Prev}AdjacentAsNumber" correspond to network topology information. In addition, MPLS-related Information Elements, such as "mplsLabelStackSection", that are useless for customers might be removed in the case of feeding Flow Records to VPN customers.

Modifying the value of Information Elements

This function modifies the value of specified Information Elements according to instructions configured by the user.

For example, this function enables IPFIX Mediator to overwrite private information with zeros or the maximum value. In particular, IP address and port number is sensitive private information. In the case of monitoring traffic trends and traffic engineering, these Information Elements are not essential factors for those purposes. In that case, this function anonymizes the relevant Information Elements to prevent a violation of privacy. If modification can anonymize some Information Elements, the function might need to report which Information Elements are anonymized. For example, "anonymizationIndicator" indicates which Information Elements have been anonymized as a bitmap, just like the "flowKeyIndicator". The anonymization method is outside the scope of this document.

3.1.3.  Exporting Process

The Exporting Process forwards Flow Records to the next Collector. The process manages the reporting Template and makes an IPFIX datagram.

In addition, the process carries out the Distributing Function as an option. If this function is enabled, the process classifies Flow Records based on the value of specified Information Elements and then exports each classified Flow Record to the appropriate Collector.

For example, the Exporting Process classifies Flow Records on the basis of the peering AS, as shown in the following figure. The set of classified Flow Records is exported to a dedicated Collector on the basis of the Peering AS.

    +-------------------------------------------+
    | IPFIX Mediator               .----------. |
    |                              |Exporting | |
    |                              |Process   | |
    |   .----------.  .---------.  |          | | PeerAS#100
Flow -->|Collecting|->|Metering |----->/------------> Collector#A
Records |Process   |  |Process  |  |   |      | | PeerAS#200
    |   '----------'  '---------'  |   /------------> Collector#B
    |                              |   |      | | PeerAS#300
    |                              |   /------------> Collector#C
    |                              |   |      | | PeerAS#400
    |                              |   /------------> Collector#D
    |                              |          | |
    |                              '----------' |
    +-------------------------------------------+

Figure B: Exporting each classified Flow Record to dedicated Collector.

3.1.4.  Storing Process

The Storing Process stores the input Flow Records from any Processes in a storage system, such as a database or flat-file system. If the storing record structure that user wants is different from the template of input Flow Records, this process selects specified Information Elements from the input Flow Records using the instruction rules. Instruction rules configured by user indicate whether the Information Elements should be stored by using a field modifier. The field modifier that indicates "keep" or "discard" is applied to Information Elements within Flow Record and IPFIX header, such as "Observation Domain ID" and "Export time". This header information MAY be used when IPFIX datagrams are made of past Flow Records. That procedure is similar to the instruction of the Aggregating Process.

When another device retrieves past Flow Records on the basis of a time period given by a user, the retrieving method can be considered in many ways. One solution is that another device gets a specified flat file from a Mediator and decodes that flat file by itself. Other solutions are that another device sends out a query command to the IPFIX Mediator through XML-RPC, SNMP, or NETCONF, and then, the IPFIX Mediator exports the specified past Flow Records. This method is outside the scope of this document.

3.2.  IPFIX Protocol Considerations

This section describes IPFIX Protocol considerations with regard to IPFIX Mediator.

3.2.1.  Export Time Issue

If the Exporting Process writes the "Export Time" of the IPFIX message when an IPFIX message leaves, an IPFIX Mediator needs to compensate for any delta time, which is the difference from "Export Time" of Information Elements contained in each Flow Record by performing calculations. An IPFIX Proxy MUST reuse the "Export Time" of received IPFIX messages from the Original Exporter.

3.2.2.  Observation Domain ID Management

The Observation Domain ID is locally unique to the Exporting Process in IPFIX Mediator. To comply with the IPFIX Protocol, the Observation Domain ID value is RECOMMENDED to be assigned a unique value per IPFIX Mediator. If an IPFIX Mediator relays an IPFIX datagram from a transport session to a transport session, IPFIX Mediator does not need to overwrite the Observation Domain ID with another value. If an IPFIX Mediator relays an IPFIX datagram from multiple transport sessions to a single transport session, IPFIX Mediator needs to overwrite the Observation Domain ID. In that case, IPFIX Mediator assigns the Observation Domain ID based on received Transport Session Information and the original Observation Domain ID. The renewed Observation Domain ID SHOULD be managed using the received Transport Session Information and original Observation Domain ID. This linkage information is available for overwriting the scope field of the Option Template.

Note
If the Metering Process aggregates input Flow Records, the value of the Observation Domain ID should be 0 to comply with the description in [I‑D.ietf‑ipfix‑protocol] (Claise, B., “IPFIX Protocol Specification,” September 2007.).

3.2.3.  Template Management

The Template ID of a generated Template SHOULD be unique on the basis of the Observation Domain ID assigned by an IPFIX Mediator. The Template ID needs to be unique on the basis of IPFIX Mediator when the Observation Domain ID is 0. If the IPFIX Mediator overwrites the received Template ID to relay a received Template or modified Template, the renewed Template ID SHOULD be managed using received Transport Session Information and the received Observation Domain ID. This linkage information is available for overwriting the scope field of an Option Template and Template handling. If IPFIX Mediator receives a "Template Withdraw Message", it SHOULD modify this message to indicate relevant Templates, and send a "Template Withdraw Message".

3.2.4.  Transport Session Management

Each session of the Collecting Process and Exporting Process should operate independently. Even if one session is reset, the status of the other session is kept current. However, Templates for resetting the Collecting Session SHOULD be withdrawn for the Exporting Session.

3.2.5.  Option Template Management

IPFIX Mediator MUST check whether the scope field is applicable, if Data Records associated with Option Templates are exported. If an IPFIX Mediator rewrites the Observation Domain ID or Template ID, these values included in scope fields SHOULD be rewritten before exporting. Instead of exporting Option Template Records and associated Data Records, Information Elements, such as sampling rate or sampling method, exported using the Option template Record from the Original Exporter could be merged in a Flow Record in an IPFIX Mediator. In that case, IPFIX Mediator MUST modify the relevant Template Record. Several sorts of received Statistics Option Template Records and associated Data Records could be exported in different ways as other Templates. In IPFIX Mediator, the Data Record associated by Statistics Option Template Records can be exported after merging its counter. In addition, Statistics Option Template Records and associated Data Records can be exported by indicating the source of the statistics data as a scope field instead of merging the counter. This method is described in Section 5. The user policy determines whether IPFIX Mediator and the above methods should export Option Templates Records and associated Data Records.

3.2.6.  Reporting of Exporter Information

If IPFIX Mediator acts as an IPFIX Firewall or IPFIX Proxy, reporting the Original Exporter IP address increases the vulnerability. On the other hand, if IPFIX Mediator acts as other devices, such as IPFIX concentrator, the Exporter IP address is important information for traffic analysis such as traffic engineering. In the case of making a traffic matrix, the Exporter IP address can indicate the ingress router of a network domain. Therefore, reporting of Exporter Information, such as Exporter IP address, is useful to identify the Original Exporter. There are various methods as follows.

An IPFIX Mediator can directly merge Exporter Information into Flow Records or use Option Templates described in Section 5. If an IPFIX Mediator receives Information Elements related to the Exporter information, IPFIX Mediator SHOULD NOT rewrite its own previous Exporter information. The IPFIX Mediator can append its own previous Exporter Information instead of rewriting. In the Collecting Process, the order of the Exporter information indicates the Original Exporter and the route of IPFIX Mediator. These methods defined by user policy determine whether IPFIX Mediator should report Exporter Information.

4.  Solution Scenarios with IPFIX Mediators

4.1.  Flexible Aggregation

An IPFIX Mediator can aggregate Flow Records in the same manner as that of IPFIX concentrator and reduce the number of Flow Records received by a Collector.

The following figure indicates a cascade connection of IPFIX Mediators. If a Collector measures a traffic matrix to obtain traffic demand, the Collector needs Flow Records of the whole network domain, but does not need detailed Flow Records. In the first step, a first level Mediator receives Flow Records from IPFIX Devices and then creates aggregated low-level Flow Records. For example, this step is prefix mask aggregation. Next, a second level Mediator receives aggregated Flow Records and aggregates them further. For example, the second step is the aggregation of the BGP next-hop address and exporter address. After this, the Collector receives high-level aggregated Flow Records and then stores them. This method enables step-by-step aggregation of Flow Records without overloading a single node.

.--------.     .--------.
|IPFIX   |     |IPFIX   |
|router#1|---->|Mediator|---.
|        |     |*1      |   |
'--------'     '--------'   |    .--------.     .---------.
                            '--->|IPFIX   |     |Traffic  |
.--------.     .--------.   .--->|Mediator|---->|Collector|
|IPFIX   |     |IPFIX   |   |    |*2      |     |         |
|router#2|---->|Mediator|---'    '--------'     '---------'
|        |     |*1      |
'--------'     '--------'

Figure C: Flexible Aggregation with cascading IPFIX Mediators.

4.2.  Distributed Aggregation

When the network is used globally, the distances between PoPs become longer, and the maintenance of a dedicated management network is very expensive. Therefore, the huge number of Flow Records has burdened management networks of global ISPs. If network operators place Mediators at each PoP, the number of Flow Records exported from each PoP can be reduced. Mediators can minimize the number of Flow Records exported to the Collector. If the Collector needs detailed information, it can retrieve Flow Records from Mediators that store original Flow Records.

A management network of a global ISP is shown in the following figure. The Mediators are located at each PoP of the network, and they collect Flow Records from routers in each PoP domain. The Mediator reduces the number of Flow Records by aggregating or filtering, so this system reduces the load of a management network.

             POP#Asia
     .--------.
   .--------. |      .---------.
 .--------. | |----->|IPFIX    |
 |IPFIX   | |------->|Mediator |----.
 |router  |---'----->|#1       |    |
 |#1      |-'        '---------'    |
 '--------'                         |
                                    |
             POP#America            |
     .--------.                     |
   .--------. |      .---------.    |     .---------.
 .--------. | |----->|IPFIX    |    '---->|Traffic  |
 |IPFIX   | |------->|Mediator |--------->|Collector|
 |router  |---'----->|#2       |    .---->|         |
 |#4      |-'        '---------'    |     '---------'
 '--------'                         |
                                    |
             POP#Europe             |
     .--------.                     |
   .--------. |      .---------.    |
 .--------. | |----->|IPFIX    |    |
 |IPFIX   | |------->|Mediator |----'
 |router  |---'----->|#3       |
 |#7      |-'        '---------'
 '--------'

Figure D: Traffic monitoring architecture in global network.

4.3.  Duplication of Flow Records

An IPFIX Mediator duplicates Flow Records to achieve redundant storage or utilizes them for several purposes. The pair of Collecting Process and Metering Process is similar to the pair of the Observation Point and Metering Process. The Collecting Process duplicates Flow Records by forwarding them to the multi-Metering Process.

Several departments in an ISP want to use the same traffic information for each intended purpose. The network design department measures the traffic matrix to obtain traffic demand. The customer service division uses traffic information for performing accounting services for each customer while network operators use traffic information for trouble shooting analysis. That situation is shown in the following figure. An IPFIX Mediator distributes Flow Records to several Collectors that have the appropriate aggregated granularity. In addition, when network operators conduct troubleshooting, past Flow Records from Mediators can be retrieved.

                                      Measurement traffic matrix.
.--------.                               .---------.
|IPFIX   |                               |Traffic  |
|router#1|----.                    .---->|Collector|
|        |    |                    |     |#1       |
'--------'    |                    |     '---------'
              |                    |  Using Accounting info.
.--------.    |     .---------.    |     .---------.
|IPFIX   |    '---->|IPFIX    |----'     |Traffic  |
|router#2|--------->|Mediator |--------->|Collector|
|        |    .---->|         |----.     |#2       |
'--------'    |     '---------'    |     '---------'
              |                    |  Using Trouble shooting.
.--------.    |                    |     .---------.
|IPFIX   |    |                    |     |Traffic  |
|router#3|----'                    '---->|Collector|
|        |                               |#3       |
'--------'                               '---------'

Figure E: Duplication of Flow Records for several purposes.

4.4.  Distribution of Flow Records

An IPFIX Mediator MAY distribute Flow Records based on the value of specified Information Elements. This function enables load balancing of Collector and sorting Flow Records without extra Collector functions. If Flow Records are used as accounting information, Mediator can distribute Flow Records to the dedicated Collector of each customer.

When network operators disclose traffic information to each customer, security or the privacy policy should be considered. In that case, the IPFIX Mediator hides private information about each customer. In addition, Mediator distributes traffic information based on RD (Route Distinguisher), ingress IF, peering AS number, or BGP next hop, which identify the customer. In the following figure, the IPFIX Mediator distributes Flow Records based on RD. The system securely allows each customer to access only their own records.

.--------.                               .---------.
|IPFIX   |                               |Traffic  |
|router#1|----.                    .---->|Collector|<===> Customer#A
|        |    |                    |     |#1       |
'--------'    |                    |     '---------'
              |                 RD=100:1
              |     .---------.    |
.--------.    '---->|IPFIX    |----'     .---------.
|IPFIX   |          |Mediator | RD=100:2 |Traffic  |
|router#2|--------->|         |--------->|Collector|<===> Customer#B
|        |          |         |          |#2       |
'--------'    .---->|         |----.     '---------'
              |     '---------'    |
              |                 RD=100:3
.--------.    |                    |     .---------.
|IPFIX   |    |                    |     |Traffic  |
|router#3|----'                    '---->|Collector|<===> Customer#C
|        |                               |#3       |
'--------'                               '---------'

Figure F: Distribution of Flow Records for each customer.

4.5.  Extraction of Suspicious Flow

An IPFIX Mediator performs filtering based on the value of specified Information Elements. Filter conditions are set depending on a suspicious flow as follows. The Collector receives the specified suspicious flow and detects an anomalous flow by simply monitoring the traffic volume of each suspicious flow.

• TCP Flow Records whose "tcpControlBits" value is set to "null"
• TCP Flow Records whose "tcpControlBits" value is set to the SYN bit only and the packet counter is only 1.
• ICMP Flow Records whose length is too long.

5.  Mediator Option Template Presentation

This section describes Option Templates that are used by IPFIX Mediators.

5.1.  Exporter Information Option Template

Each IPFIX Mediator and final destination Collector needs to know the Original Exporter and route of IPFIX Mediators. Therefore, each IPFIX Mediator informs the next Collector about previous Exporter information, which is the Exporter Information Option Template that specified the Original Exporter and the route of the IPFIX Mediator. The final destination Collector can recognize them by receiving this template. This template is composed of the following Information Elements.

• exporter{IPv4|IPv6}Address
• collector{IPv4|IPv6}Address
• exporterTransportPort
• collectorTransportPort
• collectorTransportProtocol
• observationDomainId

The Observation Domain ID of the Original Exporter or IPFIX Mediator is identified by specifying Exporter/Collector Information Elements, such as "collector{IPv4|IPv6}Address", "collectorTransportPort", "collectorTransportProtocol", "exporter{IPv4|IPv6}Address", "exporterTransportPort", and "observationDomainId". The set of "observationDomainId" and "templateId" or "observationDomainId" might be used as a scope field. Not all Information Elements are necessary. For example, the exporter{IPv4|IPv6}Address is necessary to inform the next Collector about the Original Exporter that created Flow Records. If the IPFIX Mediator receives this Template, it SHOULD not overwrite each field. The IPFIX Mediator appends its own previous Exporter information onto received Data Records specified by the Exporter Option Template and sends that information to the Collector. In this manner, the route is maintained until the final destination Collector.

The following example describes the cascade connection of IPFIX Mediators. Each Mediator informs the next Collector about previous Exporter information.

       Session#a            Session#b            Session#c
Router --------> Mediator#1 --------> Mediator#2 -------->Collector
IP:10.1.1.1      IP:10.1.1.2          IP:10.1.1.3
SrcPort:6666     DstPort:4739         DstPort:4739
ODID:10          SrcPort:7777         SrcPort:8888
                 ODID:0               ODID:0

Figure G: Cascade connection of IPFIX Mediators.

Mediator#1 or Mediator#2 sends a Data Record specified by the Exporter Option Template. The Data records are shown in Session#b or Session#c, as follows.

Session#b Data Record:

Field Count = 7
Scope Count = 1
templateId = XXX
exporterIPv4Address = 10.1.1.1
collectorIPv4Address = 10.1.1.2
collectorTransportProtocol = 132
exporterTransportPort = 6666
collectorTransportPort = 4739
observationDomainId = 10

Session#c Data Record:

Field Count = 13
Scope Count = 1
templateId = XXX
exporterIPv4Address = 10.1.1.1
collectorIPv4Address = 10.1.1.2
collectorTransportProtocol = 132
exporterTransportPort = 6666
collectorTransportPort = 4739
observationDomainId = 10
exporterIPv4Address = 10.1.1.2
collectorIPv4Address = 10.1.1.3
collectorTransportProtocol = 132
exporterTransportPort = 7777
collectorTransportPort = 4739
observationDomainId = 0

5.2.  Usage of Scope Field

An IPFIX Mediator needs to send Options Template Records and associated Data Records from the Original Exporter. However, IPFIX Mediator cannot export original Option Template Records and associated Data Records without modification because changing a session from an Exporting Process to a Collecting Process causes the scope fields to have a useless values. When an IPFIX Mediator relays the Option Template Records that included Observation Domain ID as a scope field and associated Data Records, an IPFIX Mediator uses the Exporter Information Option Template. Option Template Records that were created from an Original Exporter can use all fields of the Exporter Information Option template as multiple scope fields. Option Template Records that were created from an IPFIX Mediator can use some fields of the Exporter Information Option Template as multiple scope fields. An IPFIX Mediator needs to modify associated Data Records according to the modified Options Template Record. However, if each node uses another field, except for the Observation Domain ID, as the scope, the scope field should be considered on a case-by-case basis.

The following example describes the cascade connection of IPFIX Mediators. Router#1 and Mediator#1 export the Metering Process Statistics Option Template.

       Session#a            Session#b            Session#c
Router --------> Mediator#1 --------> Mediator#2 -------->Collector
IP:10.1.1.1      IP:10.1.1.2         IP:10.1.1.3
SrcPort:6666     DstPort:4739        DstPort:4739
ODID:10          SrcPort:7777        SrcPort:8888
                 ODID:0              ODID:0

Figure H: Cascade connection of IPFIX Mediators.

Session#c Metering Process Statistics Data Records from the Original Exporter:

Field Count = 15
Scope Count = 12
exporterIPv4Address = 10.1.1.1
collectorIPv4Address = 10.1.1.2
collectorTransportProtocol = 132
exporterTransportPort = 6666
collectorTransportPort = 4739
observationDomainId = 10
exporterIPv4Address = 10.1.1.2
collectorIPv4Address = 10.1.1.3
collectorTransportProtocol = 132
exporterTransportPort = 7777
collectorTransportPort = 4739
observationDomainId = 0
exportedMessageTotalCount
exportedFlowTotalCount
exportedOctetTotalCount

6.  Security Considerations

The IPFIX concentrator uses the IPFIX protocol. Security considerations about flow information are described in [I‑D.ietf‑ipfix‑protocol] (Claise, B., “IPFIX Protocol Specification,” September 2007.).

7.  IANA Considerations

This document has no actions for IANA.

8.  References

8.1. Normative References

8.2. Informative References

Authors' Addresses

Full Copyright Statement

Copyright © The IETF Trust (2007).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an “AS IS” basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.