Internet-Draft VN/TE Perf Monitoring March 2022
Lee, et al. Expires 8 September 2022 [Page]
Workgroup:
TEAS Working Group
Internet-Draft:
draft-ietf-teas-actn-pm-telemetry-autonomics-08
Published:
Intended Status:
Standards Track
Expires:
Authors:
Y. Lee, Ed.
Samsung Electronics
D. Dhody, Ed.
Huawei Technologies
S. Karunanithi
Huawei Technologies
R. Vilalta
CTTC
D. King
Lancaster University
D. Ceccarelli
Ericsson

YANG models for Virtual Network (VN)/TE Performance Monitoring Telemetry and Scaling Intent Autonomics

Abstract

This document provides YANG data models that describe performance monitoring telemetry and scaling intent mechanisms for TE-tunnels and Virtual Networks (VNs).

The models presented in this document allow customers to subscribe to and monitor the key performance data of the TE-tunnel or the VN. The models also provide customers with the ability to program autonomic scaling intent mechanisms on the level of TE-tunnel as well as VN.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 8 September 2022.

Table of Contents

1. Introduction

The YANG [RFC7950] model in [I-D.ietf-teas-actn-vn-yang] is used to operate customer-driven Virtual Networks (VNs) during the computation of VN, its instantiation, and its life-cycle service management and operations. YANG model in [I-D.ietf-teas-yang-te] is used to operate TE-tunnels during the tunnel instantiation, and its life-cycle management and operations.

The models presented in this draft allow the applications hosted by the customers to subscribe to and monitor their key performance data of their interest on the level of VN [I-D.ietf-teas-actn-vn-yang] or TE-tunnel [I-D.ietf-teas-yang-te]. The key characteristic of the models presented in this document is a top-down programmability that allows the applications hosted by the customers to subscribe to and monitor key performance data of their interest and autonomic scaling intent mechanism on the level of VN as well as TE-tunnel.

According to the classification of [RFC8309], the YANG data models presented in this document can be classified as customer service models. These can be mapped to the CMI (Customer Network Controller (CNC)- Multi-Domain Service Coordinator (MSDC) interface) of ACTN [RFC8453].

[RFC8233] describes key network performance data to be considered for end-to-end path computation in TE networks. The services provided can be optimized to meet the requirements (such as traffic patterns, quality, and reliability) of the applications hosted by the customers.

This document provides YANG data models generically applicable to any VN/TE-Tunnel service clients to provide an ability to program their customized performance monitoring subscription and publication data models and automatic scaling in/out intent data models. These models can be utilized by a client network controller to initiate the capabilities to a TE network controller communicating with the client controller via a NETCONF [RFC8341] or a RESTCONF [RFC8040] interface.

The term performance monitoring is used in this document in a different from how the term has been used in TE networks for many years. Performance monitoring in this document refers to subscription and publication of streaming telemetry data. Subscription is initiated by the client (e.g., CNC) while publication is provided by the network (e.g., MDSC/Provisioning Network Controller (PNC)) based on the client's subscription. As the scope of performance monitoring in this document is telemetry data on the level of a client's VN or TE-tunnel, the entity interfacing to the client (e.g., MDSC) has to provide VN or TE-tunnel level information. This requires the controller to have the capability to derive VN or TE-tunnel level performance data based on lower-level data collected via PM counters in the Network Elements (NE). How the controller entity derives such customized level data (i.e., VN or TE-tunnel level) is out of the scope of this document.

The data model includes configuration and state data according to the Network Management Datastore Architecture (NMDA) [RFC8342].

1.1. Terminology

Refer to [RFC8453], [RFC7926], and [RFC8309] for the key terms used in this document.

Scaling: This refers to the network's ability to re-shape its own resources. "Scale out" refers to improve network performance by increasing the allocated resources, while "scale in" refers to decreasing the allocated resources, typically because the existing resources are unnecessary.

Scaling Intent: Scaling intent is used to declare scaling conditions. Specifically, scaling intent refers to how the client programs or configures conditions that will be applied to their key performance data to trigger either scaling out or scaling in. Various conditions can be set for scaling intent on either VN or TE-tunnel level.

Network Autonomics: This refers to the network automation capability that allows a client to initiate scaling intent mechanisms and provides the client with the status of the adjusted network resources based on the client's scaling intent in an automated fashion.

1.2. Tree Diagram

A simplified graphical representation of the data model is used in Section 4 and Section 6 of this document. The meaning of the symbols in these diagrams is defined in [RFC8340].

1.3. Prefixes in Data Node Names

In this document, names of data nodes and other data model objects are prefixed using the standard prefix associated with the corresponding YANG imported modules, as shown in Table 1.

Table 1: Prefixes and corresponding YANG modules
Prefix YANG module Reference
te ietf-te [I-D.ietf-teas-yang-te]
te-types ietf-te-types [RFC8776]
te-tel ietf-te-telemetry [RFCXXXX]
vn ietf-vn [I-D.ietf-teas-actn-vn-yang]
vn-tel ietf-vn-telemetry [RFCXXXX]

Note: The RFC Editor is requested to replace XXXX with the number assigned to the RFC once this draft becomes an RFC, and to remove this note.

Further, the following additional documents are referenced in the model defined in this document -

  • [RFC7471] - OSPF Traffic Engineering (TE) Metric Extensions.
  • [RFC8570] - IS-IS Traffic Engineering (TE) Metric Extensions.
  • [RFC7823] - Performance-Based Path Selection for Explicitly Routed Label Switched Paths (LSPs) Using TE Metric Extensions.

2. Use-Cases

There is a need for real-time (or semi-real-time) traffic monitoring of the network to optimize the network and the traffic distribution. Figure 1 shows the high-level workflow for dynamic service control based on traffic monitoring.

   +----------------------------------------------+
   | Client   +-----------------------------+     |
   |          | Dynamic Service Control APP |     |
   |          +-----------------------------+     |
   +----------------------------------------------+
   1.Traffic|  /|\4.Traffic            | /|\
   Monitor &|   | Monitor              |  | 8.Traffic
   Optimize |   | Result     5.Service |  | modify &
   Policy   |   |              modify &|  | optimize
           \|/  |        optimize Req.\|/ | result
   +----------------------------------------------+
   | Orchestrator                                 |
   |    +-------------------------------+         |
   |    |Dynamic Service Control Agent  |         |
   |    +-------------------------------+         |
   |    +---------------+ +-------------------+   |
   |    | Flow Optimize | | vConnection Agent |   |
   |    +---------------+ +-------------------+   |
   +----------------------------------------------+
   2. Path |   /|\3.Traffic            | /|\
   Monitor |    | Monitor              |  |7.Path
   Request |    | Result      6.Path   |  | modify &
           |    |             modify & |  | optimize
          \|/   |        optimize Req.\|/ | result
   +----------------------------------------------+
   | Network SDN Controller                       |
   |  +----------------------+ +-----------------+|
   |  | Network Provisioning | |Abstract Topology||
   |  +----------------------+ +-----------------+|
   |  +------------------+ +--------------------+ |
   |  |Network Monitoring| |Physical Topology DB| |
   |  +------------------+ +--------------------+ |
   +----------------------------------------------+

APP: Application
DB: Database
Req: Request
Figure 1: Workflow for dynamic service control based on traffic monitoring

Some of the key points are as follows:

3. Design of the Data Models

This document describes two YANG models:

(i)

TE Telemetry Model which provides the TE-Tunnel level of

performance monitoring mechanism and scaling intent mechanism that allows scale in/out programming by the customer. (See Section 3.1 & Section 7.1 for details).

(ii)

VN Telemetry Model which provides the VN level of the

aggregated performance monitoring mechanism and scaling intent mechanism that allows scale in/out programming by the customer (See Section 3.2 & Section 7.2 for details).

3.1. TE Telemetry Model

This model describes the performance telemetry for the TE tunnel. The telemetry data is augmented to the TE tunnel. This model also allows autonomic traffic engineering scaling intent configuration mechanism on the TE-tunnel level. Various conditions can be set for auto-scaling based on the telemetry data (See Section 5 for details)

As shown in Figure 2, the TE Telemetry Model augments the TE-Tunnel Model to enhance TE performance monitoring capability. This monitoring capability will facilitate re-optimization and reconfiguration of TE tunnels based on the performance monitoring data collected via the TE Telemetry YANG model.

             +------------+          +--------------+
             |  TE-Tunnel |          |      TE      |
             |   Model    |<---------|  Telemetry   |
             +------------+ augments |     Model    |
                                     +--------------+
Figure 2: TE Telemetry Model Relationship

3.2. VN Telemetry Model

As shown in Figure 3, the VN Telemetry Model augments the basic VN model to enhance VN monitoring capability. This monitoring capability will facilitate re-optimization and reconfiguration of VNs based on the performance monitoring data collected via the VN Telemetry YANG model. This model also imports TE telemetry model to reuse the groupings.

             +----------+          +--------------+
             |    VN    | augments |      VN      |
             |   Model  |<---------|   Telemetry  |
             +----------+          |     Model    |
                                   +--------------+
                                          |
                                          | imports
                                          v
                                   +--------------+
                                   |      TE      |
                                   |   Telemetry  |
                                   |     Model    |
                                   +--------------+
Figure 3: VN Telemetry Model Relationships

This model describes the performance telemetry for the VN model. The telemetry data is augmented to the VN model at the VN Level as well as at the individual VN member level. This model also allows autonomic traffic engineering scaling intent configuration mechanism on the VN level. Scale in/out criteria might be used for network autonomics in order for the controller to react to a certain set of variations in monitored parameters (See Section 4 for illustrations).

Moreover, this model also provides a mechanism to define aggregated VN telemetry parameters as a grouping of underlying VN-member level telemetry parameters. This is unique to the VN model as a VN is made up of multiple VN-members and further each VN-member could be set across multiple TE tunnels. Grouping operation (such as maximum, mean) could be set at the time of configuration. For example, if "maximum" grouping operation is used for delay at the VN level, the VN telemetry data is reported as the maximum of {delay_vn_member_1, delay_vn_member_2,.. delay_vn_member_N}. Thus, this telemetry aggregation mechanism allows the aggregation (or grouping) of a certain common set of telemetry values under a grouping operation. This can also be done at the VN-member level to suggest how the end-to-end (E2E) telemetry be inferred from the per domain tunnels created and monitored by PNCs. The Figure 4 provides an example interaction.

  +------------------------------------------------------------+
  |                      Client                                |
  |                                                            |
  +------------------------------------------------------------+
  1.Client sets the      |   /|\   2. Orchestrator pushes:
  grouping op, and       |    |
  subscribes to the      |    |    VN level telemetry for
  VN level telemetry for |    |    - VN Utilized-bw-percentage
  Delay and              |    |       (Minimum across VN Members)
  Utilized-bw-pecentage  |    |    - VN Delay (Maximum across VN
                        \|/   |     Members)
   +------------------------------------------------------------+
   | Orchestrator                                               |
   |                                                            |
   +------------------------------------------------------------+
Figure 4: TE Telemetry Model Interactions

3.3. VPN Service Performance Monitoring

The YANG model in [I-D.ietf-opsawg-yang-vpn-service-pm] provides network performance monitoring (PM) and VPN service performance monitoring that can be used to monitor and manage network performance on the topology at higher layer or the service topology between VPN sites. Thus the YANG models in this document could be used along side with ietf-network-vpn-pm to understand and correlate the performance monitoring at the VPN service and the underlying TE level.

4. Autonomic Scaling Intent Mechanism

The scaling intent configuration mechanism allows the client to configure automatic scale-in and scale-out mechanisms on both the TE-tunnel and the VN level. Various conditions can be set for auto-scaling based on the PM telemetry data.

There are a number of parameters involved in the mechanism:

The tree in Figure 5 is a part of ietf-te-telemetry tree whose model is presented in full detail in Sections 6 & 7.

module: ietf-te-telemetry
  augment /te:te/te:tunnels/te:tunnel:
    +--rw te-scaling-intent
    |  +--rw scale-in-intent
    |  |  +--rw threshold-time?      uint32
    |  |  +--rw cooldown-time?       uint32
    |  |  +--rw scaling-condition* [performance-type]
    |  |  |  +--rw performance-type           identityref
    |  |  |  +--rw threshold-value?           string
    |  |  |  +--rw scale-in-operation-type?
    |  |  |          scaling-criteria-operation
    |  |  +--rw scale-in-op?         identityref
    |  |  +--rw scale?               string
    |  +--rw scale-out-intent
    |     +--rw threshold-time?      uint32
    |     +--rw cooldown-time?       uint32
    |     +--rw scaling-condition* [performance-type]
    |     |  +--rw performance-type            identityref
    |     |  +--rw threshold-value?            string
    |     |  +--rw scale-out-operation-type?
    |     |          scaling-criteria-operation
    |     +--rw scale-out-op?        identityref
    |     +--rw scale?               string
Figure 5: The scaling intent

Let's say the client wants to set the scaling out operation based on two performance-types (e.g., two-way-delay and utilized-bandwidth for a te-tunnel), it can be done as follows:

In the scaling condition's list, the following two components can be set:

List 1: Scaling Condition for Two-way-delay

List 2: Scaling Condition for Utilized bandwidth

5. Notification

This model does not define specific notifications. To enable notifications, the mechanism defined in [RFC8641] and [RFC8640] can be used. This mechanism currently allows the user to:

5.1. YANG Push Subscription Examples

[RFC8641] allows subscriber applications to request a continuous, customized stream of updates from a YANG datastore.

The example in Figure 6 shows the way for a client to subscribe to the telemetry information for a particular tunnel (Tunnel1). The telemetry parameter that the client is interested in is one-way- delay.

<netconf:rpc netconf:message-id="101"
    xmlns:netconf="urn:ietf:params:xml:ns:netconf:base:1.0">
    <establish-subscription
       xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push:1.0">
       <filter netconf:type="subtree">
          <te xmlns="urn:ietf:params:xml:ns:yang:ietf-te">
             <tunnels>
                <tunnel>
                  <name>Tunnel1</name>
                    <te-telemetry xmlns="urn:ietf:params:xml:ns:yang:
                                         ietf-te-telemetry">
                        <performance-metrics-one-way>
                           <one-way-delay/>
                        </performance-metrics-one-way>
                     </te-telemetry>
                  </tunnel>
              </tunnels>
          </te>
       </filter>
       <period>500</period>
       <encoding>encode-xml</encoding>
    </establish-subscription>
 </netconf:rpc>
Figure 6: TE Tunnel Subscription Example

The example in Figure 7 shows the way for a client to subscribe to the telemetry information for all VNs. The telemetry parameter that the client is interested in is one-way-delay and one-way-utilized- bandwidth.

<netconf:rpc netconf:message-id="101"
    xmlns:netconf="urn:ietf:params:xml:ns:netconf:base:1.0">
  <establish-subscription
     xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-push:1.0">
     <filter netconf:type="subtree">
        <virtual-network xmlns="urn:ietf:params:xml:ns:yang:ietf-vn">
           <vn>
                <vn-id/>
                <vn-telemetry xmlns="urn:ietf:params:xml:ns:yang:
                                      ietf-vn-telemetry">
                  <params>
                    <performance-metrics-one-way>
                      <one-way-delay/>
                      <one-way-utilized-bandwidth/>
                    </performance-metrics-one-way>
                  </params>
                </vn-telemetry>
            </vn>
        </virtual-network>
     </filter>
     <period>500</period>
  </establish-subscription>
</netconf:rpc>
Figure 7: VN Subscription Example

5.2. Scaling Examples

The example in Figure 8 shows the way to configure a TE tunnel with the scaling-out intent to re-optimize when the the scaling condition of two-way-delay crossing 100 milliseconds (100000 microseconds) for a threshold of 1 min (60000 milliseconds).

<edit-config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
   <target>
     <running/>
   </target>
   <config>
     <te xmlns="urn:ietf:params:xml:ns:yang:ietf-te">
         <tunnels>
            <tunnel>
              <name>Tunnel1</name>
                <te-scaling-intent
                  xmlns="urn:ietf:params:xml:ns:yang:
                         ietf-te-telemetry">
                    <scale-out-intent>
                       <threshold-time>
                        60000
                      </threshold-time>
                      <scaling-condition>
                        <performance-type>
                          two-way-delay
                        </performance-type>
                        <threshold-value>
                          100000
                        </threshold-value>
                        <scale-out-op>
                          re-optimize
                        </scale-out-op>
                      </scaling-condition>
                    </scale-out-intent>
                 </te-scaling-intent>
              </tunnel>
          </tunnels>
     </te>
   </config>
</edit-config>
Figure 8: TE Tunnel Scaling Example

The example in Figure 9 shows the way to configure a VN with the scaling-in intent to reduce bandwidth when the the scaling condition of two-way-delay crossing 100 milliseconds (100000 microseconds) for a threshold of 1 min (60000 milliseconds).

<edit-config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
   <target>
     <running/>
   </target>
   <config>
     <virtual-network xmlns="urn:ietf:params:xml:ns:yang:ietf-vn">
          <vn>
              <vn-id>VN1</vn-id>
                <vn-scaling-intent
                  xmlns="urn:ietf:params:xml:ns:yang:
                         ietf-vn-telemetry">
                    <scale-in-intent>
                      <threshold-time>60000</threshold-time>
                      <scaling-condition>
                        <performance-type>
                          utilized-percentage
                        </performance-type>
                        <threshold-value>
                          50
                        </threshold-value>
                        <scale-in-op>
                          scale-capacity-down
                        </scale-in-op>
                      </scaling-condition>
                    </scale-in-intent>
              </vn-scaling-intent>
          </vn>
     </virtual-network>
   </config>
</edit-config>
Figure 9: VN Scaling Example

The example in Figure 10 shows the way to configure a grouping operation at the VN level to require that the VN level one-way-delay needs to be the reported as the max of the one-way-delay at the VN-member level, where as the utilized-percentage is the mean.

<edit-config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
   <target>
     <running/>
   </target>
   <config>
     <virtual-network xmlns="urn:ietf:params:xml:ns:yang:ietf-vn">
          <vn>
              <vn-id>VN1</vn-id>
              <vn-telemetry
                xmlns="urn:ietf:params:xml:ns:yang:
                       ietf-vn-telemetry">
                <operation>
                  <performance-type>
                    one-way-delay
                  </performance-type>
                  <grouping-operation>
                    maximum
                  </grouping-operation>
                </operation>
                <operation>
                  <performance-type>
                    utilized-percentage
                  </performance-type>
                  <grouping-operation>
                    mean
                  </grouping-operation>
                </operation>
              </vn-telemetry>
          </vn>
     </virtual-network>
   </config>
</edit-config>
Figure 10: VN Grouping Operation Example

6. YANG Data Tree



module: ietf-te-telemetry
  augment /te:te/te:tunnels/te:tunnel:
    +--rw te-scaling-intent
    |  +--rw scale-in-intent
    |  |  +--rw threshold-time?      uint32
    |  |  +--rw cooldown-time?       uint32
    |  |  +--rw scaling-condition* [performance-type]
    |  |  |  +--rw performance-type           identityref
    |  |  |  +--rw threshold-value?           string
    |  |  |  +--rw scale-in-operation-type?
    |  |  |          scaling-criteria-operation
    |  |  +--rw scale-in-op?         identityref
    |  |  +--rw scale?               string
    |  +--rw scale-out-intent
    |     +--rw threshold-time?      uint32
    |     +--rw cooldown-time?       uint32
    |     +--rw scaling-condition* [performance-type]
    |     |  +--rw performance-type            identityref
    |     |  +--rw threshold-value?            string
    |     |  +--rw scale-out-operation-type?
    |     |          scaling-criteria-operation
    |     +--rw scale-out-op?        identityref
    |     +--rw scale?               string
    +--ro te-telemetry
       +--ro id?                            telemetry-id
       +--ro performance-metrics-one-way
       |  +--ro one-way-delay?                           uint32
       |  +--ro one-way-delay-normality?
       |  |       te-types:performance-metrics-normality
       |  +--ro one-way-residual-bandwidth?
       |  |       rt-types:bandwidth-ieee-float32
       |  +--ro one-way-residual-bandwidth-normality?
       |  |       te-types:performance-metrics-normality
       |  +--ro one-way-available-bandwidth?
       |  |       rt-types:bandwidth-ieee-float32
       |  +--ro one-way-available-bandwidth-normality?
       |  |       te-types:performance-metrics-normality
       |  +--ro one-way-utilized-bandwidth?
       |  |       rt-types:bandwidth-ieee-float32
       |  +--ro one-way-utilized-bandwidth-normality?
       |          te-types:performance-metrics-normality
       +--ro performance-metrics-two-way
          +--ro two-way-delay?             uint32
          +--ro two-way-delay-normality?
                  te-types:performance-metrics-normality


Figure 11: ietf-te-telemetry YANG model tree

module: ietf-vn-telemetry
  augment /vn:virtual-network/vn:vn:
    +--rw vn-scaling-intent
    |  +--rw scale-in-intent
    |  |  +--rw threshold-time?      uint32
    |  |  +--rw cooldown-time?       uint32
    |  |  +--rw scaling-condition* [performance-type]
    |  |  |  +--rw performance-type           identityref
    |  |  |  +--rw threshold-value?           string
    |  |  |  +--rw scale-in-operation-type?
    |  |  |          scaling-criteria-operation
    |  |  +--rw scale-in-op?         identityref
    |  |  +--rw scale?               string
    |  +--rw scale-out-intent
    |     +--rw threshold-time?      uint32
    |     +--rw cooldown-time?       uint32
    |     +--rw scaling-condition* [performance-type]
    |     |  +--rw performance-type            identityref
    |     |  +--rw threshold-value?            string
    |     |  +--rw scale-out-operation-type?
    |     |          scaling-criteria-operation
    |     +--rw scale-out-op?        identityref
    |     +--rw scale?               string
    +--rw vn-telemetry
       +--ro params
       |  +--ro performance-metrics-one-way
       |  |  +--ro one-way-delay?                           uint32
       |  |  +--ro one-way-delay-normality?
       |  |  |       te-types:performance-metrics-normality
       |  |  +--ro one-way-residual-bandwidth?
       |  |  |       rt-types:bandwidth-ieee-float32
       |  |  +--ro one-way-residual-bandwidth-normality?
       |  |  |       te-types:performance-metrics-normality
       |  |  +--ro one-way-available-bandwidth?
       |  |  |       rt-types:bandwidth-ieee-float32
       |  |  +--ro one-way-available-bandwidth-normality?
       |  |  |       te-types:performance-metrics-normality
       |  |  +--ro one-way-utilized-bandwidth?
       |  |  |       rt-types:bandwidth-ieee-float32
       |  |  +--ro one-way-utilized-bandwidth-normality?
       |  |          te-types:performance-metrics-normality
       |  +--ro performance-metrics-two-way
       |     +--ro two-way-delay?             uint32
       |     +--ro two-way-delay-normality?
       |             te-types:performance-metrics-normality
       +--rw operation* [performance-type]
          +--rw performance-type      identityref
          +--rw grouping-operation?   identityref
  augment /vn:virtual-network/vn:vn/vn:vn-member:
    +--rw vn-member-telemetry
       +--ro params
       |  +--ro performance-metrics-one-way
       |  |  +--ro one-way-delay?                           uint32
       |  |  +--ro one-way-delay-normality?
       |  |  |       te-types:performance-metrics-normality
       |  |  +--ro one-way-residual-bandwidth?
       |  |  |       rt-types:bandwidth-ieee-float32
       |  |  +--ro one-way-residual-bandwidth-normality?
       |  |  |       te-types:performance-metrics-normality
       |  |  +--ro one-way-available-bandwidth?
       |  |  |       rt-types:bandwidth-ieee-float32
       |  |  +--ro one-way-available-bandwidth-normality?
       |  |  |       te-types:performance-metrics-normality
       |  |  +--ro one-way-utilized-bandwidth?
       |  |  |       rt-types:bandwidth-ieee-float32
       |  |  +--ro one-way-utilized-bandwidth-normality?
       |  |          te-types:performance-metrics-normality
       |  +--ro performance-metrics-two-way
       |  |  +--ro two-way-delay?             uint32
       |  |  +--ro two-way-delay-normality?
       |  |          te-types:performance-metrics-normality
       |  +--ro te-grouped-params*
       |          -> /te:te/tunnels/tunnel/te-tel:te-telemetry/id
       +--rw operation* [performance-type]
          +--rw performance-type      identityref
          +--rw grouping-operation?   identityref

Figure 12: ietf-vn-telemetry YANG model tree

7. YANG Data Model

7.1. ietf-te-telemetry model

The YANG code is as follows:

<CODE BEGINS> file "ietf-te-telemetry@2022-03-07.yang"

module ietf-te-telemetry {
  yang-version 1.1;
  namespace "urn:ietf:params:xml:ns:yang:ietf-te-telemetry";
  prefix te-tel;

  /* Import TE */

  import ietf-te {
    prefix te;
    reference
      "I-D.ietf-teas-yang-te: A YANG Data Model for Traffic
       Engineering Tunnels and Interfaces";
  }

  /* Import TE Common types */

  import ietf-te-types {
    prefix te-types;
    reference
      "RFC 8776: Common YANG Data Types for Traffic Engineering";
  }

  organization
    "IETF Traffic Engineering Architecture and Signaling (TEAS)
     Working Group";
  contact
    "WG Web:  <https://datatracker.ietf.org/wg/teas/about/>
     WG List: <mailto:teas@ietf.org>
     Editor:  Young Lee <younglee.tx@gmail.com>
              Dhruv Dhody <dhruv.ietf@gmail.com>";
  description
    "This module describes YANG data model for performance
     monitoring telemetry for te tunnels.

     Copyright (c) 2022 IETF Trust and the persons identified as
     authors of the code.  All rights reserved.

     Redistribution and use in source and binary forms, with or
     without modification, is permitted pursuant to, and subject to
     the license terms contained in, the Revised BSD License set
     forth in Section 4.c of the IETF Trust's Legal Provisions
     Relating to IETF Documents
     (https://trustee.ietf.org/license-info).

     This version of this YANG module is part of RFC XXXX; see the
     RFC itself for full legal notices.";

  /* Note: The RFC Editor will replace XXXX with the number
     assigned to the RFC once draft-ietf-teas-pm-telemetry-
     autonomics becomes an RFC.*/

  revision 2022-03-07 {
    description
      "Initial revision.";
    reference
      "RFC XXXX: YANG models for VN/TE Performance Monitoring
       Telemetry and Scaling Intent Autonomics";
  }

  identity telemetry-param-type {
    description
      "Base identity for telemetry param types";
  }

  identity one-way-delay {
    base telemetry-param-type;
    description
      "To specify average Delay in one (forward) direction.

       At the VN level, it is the max delay of the VN-members.

       The threshold-value for this type is interpreted as
       microseconds.";
    reference
      "RFC 7471: OSPF Traffic Engineering (TE) Metric Extensions.
       RFC 8570: IS-IS Traffic Engineering (TE) Metric Extensions.
       RFC 7823: Performance-Based Path Selection for Explicitly
       Routed Label Switched Paths (LSPs) Using TE Metric
       Extensions";
  }

  identity two-way-delay {
    base telemetry-param-type;
    description
      "To specify average Delay in both (forward and reverse)
       directions.

       At the VN level, it is the max delay of the VN-members.

       The threshold-value for this type is interpreted as
       microseconds.";
    reference
      "RFC 7471: OSPF Traffic Engineering (TE) Metric Extensions.
       RFC 8570: IS-IS Traffic Engineering (TE) Metric Extensions.
       RFC 7823: Performance-Based Path Selection for Explicitly
       Routed Label Switched Paths (LSPs) Using TE Metric
       Extensions";
  }

  identity one-way-delay-variation {
    base telemetry-param-type;
    description
      "To specify average Delay Variation in one (forward) direction.

       At the VN level, it is the max delay variation of the
       VN-members.

       The threshold-value for this type is interpreted as
       microseconds.";
    reference
      "RFC 7471: OSPF Traffic Engineering (TE) Metric Extensions.
       RFC 8570: IS-IS Traffic Engineering (TE) Metric Extensions.
       RFC 7823: Performance-Based Path Selection for Explicitly
       Routed Label Switched Paths (LSPs) Using TE Metric
       Extensions";
  }

  identity two-way-delay-variation {
    base telemetry-param-type;
    description
      "To specify average Delay Variation in both (forward and
       reverse) directions.

       At the VN level, it is the max delay variation of the
       VN-members.

       The threshold-value for this type is interpreted as
       microseconds.";
    reference
      "RFC 7471: OSPF Traffic Engineering (TE) Metric Extensions.
       RFC 8570: IS-IS Traffic Engineering (TE) Metric Extensions.
       RFC 7823: Performance-Based Path Selection for Explicitly
       Routed Label Switched Paths (LSPs) Using TE Metric
       Extensions";
  }

  identity utilized-bandwidth {
    base telemetry-param-type;
    description
      "To specify utilized bandwidth over the specified source
       and destination.

       The threshold-value for this type is interpreted as
       bytes per second.";
    reference
      "RFC 7471: OSPF Traffic Engineering (TE) Metric Extensions.
       RFC 8570: IS-IS Traffic Engineering (TE) Metric Extensions.
       RFC 7823: Performance-Based Path Selection for Explicitly
       Routed Label Switched Paths (LSPs) Using TE Metric
       Extensions";
  }

  identity utilized-percentage {
    base telemetry-param-type;
    description
      "To specify utilization percentage of the entity
       (e.g., tunnel, link, etc.)";
  }

  identity scale-op {
    description
      "Base identity for scaling operation";
  }

  identity scale-capacity-up {
    base scale-op;
    description
      "Scale up the bandwidth capacity";
  }

  identity scale-capacity-down {
    base scale-op;
    description
      "Scale down the bandwidth capacity";
  }

  /* Typedef */

  typedef telemetry-id {
    type string;
    description
      "Identifier for the telemetry data.";
  }

  typedef scaling-criteria-operation {
    type enumeration {
      enum AND {
        description
          "AND operation";
      }
      enum OR {
        description
          "OR operation";
      }
    }
    description
      "Operations to analize list of scaling criterias";
  }

  grouping scaling-duration {
    description
      "Base scaling criteria durations";
    leaf threshold-time {
      type uint32;
      units "seconds";
      description
        "The duration for which the criteria must hold true";
    }
    leaf cooldown-time {
      type uint32;
      units "seconds";
      description
        "The duration after a scaling-in/scaling-out action has been
         triggered, for which there will be no further operation";
    }
  }

  grouping scaling-criteria {
    description
      "Grouping for scaling criteria";
    leaf performance-type {
      type identityref {
        base telemetry-param-type;
      }
      description
        "Reference to the tunnel level telemetry type";
    }
    leaf threshold-value {
      type string;
      description
        "Scaling threshold for the telemetry parameter type.";
    }
  }

  grouping scaling-in-intent {
    description
      "Basic scaling in intent";
    uses scaling-duration;
    list scaling-condition {
      key "performance-type";
      description
        "Scaling conditions";
      uses scaling-criteria;
      leaf scale-in-operation-type {
        type scaling-criteria-operation;
        default "AND";
        description
          "Operation to be applied to check between scaling criterias
           to check if the scale in threshold condition has been met.
           Defaults to AND";
      }
    }
    leaf scale-in-op {
      type identityref {
        base scale-op;
      }
      default "scale-capacity-down";
      description
        "The scaling operation to be performed when scaling condition
         is met";
    }
    leaf scale {
      type string;
      description
        "Additional scaling-by information to be interpritted as per
         the scale-in-op.";
    }
  }

  grouping scaling-out-intent {
    description
      "Basic scaling out intent";
    uses scaling-duration;
    list scaling-condition {
      key "performance-type";
      description
        "Scaling conditions";
      uses scaling-criteria;
      leaf scale-out-operation-type {
        type scaling-criteria-operation;
        default "OR";
        description
          "Operation to be applied to check between scaling criterias
           to check if the scale out threshold condition has been met.
           Defauls to OR";
      }
    }
    leaf scale-out-op {
      type identityref {
        base scale-op;
      }
      default "scale-capacity-up";
      description
        "The scaling operation to be performed when scaling condition
         is met";
    }
    leaf scale {
      type string;
      description
        "Additional scaling-by information to be interpritted as per
         the scale-out-op.";
    }
  }

  augment "/te:te/te:tunnels/te:tunnel" {
    description
      "Augmentation parameters for config scaling-criteria TE
       tunnel topologies. Scale in/out criteria might be used
       for network autonomics in order the controller to react
       to a certain set of monitored params.";
    container te-scaling-intent {
      description
        "The scaling intent";
      container scale-in-intent {
        description
          "scale-in";
        uses scaling-in-intent;
      }
      container scale-out-intent {
        description
          "scale-out";
        uses scaling-out-intent;
      }
    }
    container te-telemetry {
      config false;
      description
        "Telemetry Data";
      leaf id {
        type telemetry-id;
        description
          "ID of telemetry data used for easy reference";
      }
      uses te-types:performance-metrics-attributes;
    }
  }
}


<CODE ENDS>

7.2. ietf-vn-telemetry model

The YANG code is as follows:

<CODE BEGINS> file "ietf-vn-telemetry@2022-03-07.yang"

module ietf-vn-telemetry {
  yang-version 1.1;
  namespace "urn:ietf:params:xml:ns:yang:ietf-vn-telemetry";
  prefix vn-tel;

  /* Import VN */

  import ietf-vn {
    prefix vn;
    reference
      "I-D.ietf-teas-actn-vn-yang: A YANG Data Model for VN
       Operation";
  }

  /* Import TE */

  import ietf-te {
    prefix te;
    reference
      "I-D.ietf-teas-yang-te: A YANG Data Model for Traffic
       Engineering Tunnels and Interfaces";
  }

  /* Import TE Common types */

  import ietf-te-types {
    prefix te-types;
    reference
      "RFC 8776: Common YANG Data Types for Traffic Engineering";
  }

  /* Import TE Telemetry */

  import ietf-te-telemetry {
    prefix te-tel;
    reference
      "RFC XXXX: YANG models for VN/TE Performance Monitoring
       Telemetry and Scaling Intent Autonomics";
  }

  /* Note: The RFC Editor will replace XXXX with the number
     assigned to this draft.*/

  organization
    "IETF Traffic Engineering Architecture and Signaling (TEAS)
     Working Group";
  contact
    "WG Web:  <https://datatracker.ietf.org/wg/teas/about/>
     WG List: <mailto:teas@ietf.org>
     Editor:  Young Lee <younglee.tx@gmail.com>
              Dhruv Dhody <dhruv.ietf@gmail.com>";
  description
    "This module describes YANG data models for performance
     monitoring telemetry for Virtual Network (VN).

     Copyright (c) 2022 IETF Trust and the persons identified as
     authors of the code.  All rights reserved.

     Redistribution and use in source and binary forms, with or
     without modification, is permitted pursuant to, and subject to
     the license terms contained in, the Revised BSD License set
     forth in Section 4.c of the IETF Trust's Legal Provisions
     Relating to IETF Documents
     (https://trustee.ietf.org/license-info).

     This version of this YANG module is part of RFC XXXX; see the
     RFC itself for full legal notices.";

  /* Note: The RFC Editor will replace XXXX with the number
     assigned to the RFC once draft-lee-teas-pm-telemetry-
     autonomics becomes an RFC.*/

  revision 2022-03-07 {
    description
      "Initial revision.";
    reference
      "RFC XXXX: YANG models for VN/TE Performance Monitoring
       Telemetry and Scaling Intent Autonomics";
  }

  identity grouping-op {
    description
      "Base identity for grouping-operation";
  }

  identity minimum {
    base grouping-op;
    description
      "Select the minimum of the monitored parameters";
  }

  identity maximum {
    base grouping-op;
    description
      "The maximum of the monitored parameters";
  }

  identity mean {
    base grouping-op;
    description
      "The mean of the monitored parameters";
  }

  identity standard-deviation {
    base grouping-op;
    description
      "The standard deviation of the monitored parameters";
  }

  identity sum {
    base grouping-op;
    description
      "The sum of the monitored parameters";
  }

  identity and {
    base grouping-op;
    description
      "Logical AND operation";
  }

  identity or {
    base grouping-op;
    description
      "Logical OR operation";
  }

  grouping grouping-operation {
    list operation {
      key "performance-type";
      leaf performance-type {
        type identityref {
          base te-tel:telemetry-param-type;
        }
        description
          "Reference to the tunnel level telemetry type";
      }
      leaf grouping-operation {
        type identityref {
          base grouping-op;
        }
        description
          "describes the operation to apply to the te-grouped-params";
      }
      description
        "Grouping operation for each performance-type";
    }
    description
      "Grouping operation for each performance-type";
  }

  augment "/vn:virtual-network/vn:vn" {
    description
      "Augmentation parameters for state TE VN topologies.";
    container vn-scaling-intent {
      description
        "scaling intent";
      container scale-in-intent {
        description
          "VN scale-in";
        uses te-tel:scaling-in-intent;
      }
      container scale-out-intent {
        description
          "VN scale-out";
        uses te-tel:scaling-out-intent;
      }
    }
    container vn-telemetry {
      description
        "VN telemetry params";
      container params {
        config false;
        description
          "Read-only telemetry parameters";
        uses te-types:performance-metrics-attributes;
      }
      uses grouping-operation;
    }
  }

  augment "/vn:virtual-network/vn:vn/vn:vn-member" {
    description
      "Augmentation parameters for state TE vn member topologies.";
    container vn-member-telemetry {
      description
        "VN member telemetry params";
      container params {
        config false;
        description
          "Read-only telemetry parameters";
        uses te-types:performance-metrics-attributes;
        leaf-list te-grouped-params {
          type leafref {
            path "/te:te/te:tunnels/te:tunnel/"
               + "te-tel:te-telemetry/te-tel:id";
          }
          description
            "A list of underlying TE parameters that form the
             VN-member";
        }
      }
      uses grouping-operation;
    }
  }
}


<CODE ENDS>

8. Security Considerations

The YANG modules specified in this document defines a schema for data that is designed to be accessed via network management protocols such as NETCONF [RFC6241] or RESTCONF [RFC8040]. The lowest NETCONF layer is the secure transport layer, and the mandatory-to-implement secure transport is Secure Shell (SSH) [RFC6242]. The lowest RESTCONF layer is HTTPS, and the mandatory-to-implement secure transport is TLS [RFC8446].

The Network Configuration Access Control Model (NACM) [RFC8341] provides the means to restrict access for particular NETCONF or RESTCONF users to a preconfigured subset of all available NETCONF or RESTCONF protocol operations and content.

There are a number of data nodes defined in this YANG module that are writable/creatable/deletable (i.e., config true, which is the default). These data nodes may be considered sensitive or vulnerable in some network environments. Write operations (e.g., edit-config) to these data nodes without proper protection can have a negative effect on network operations. These are the subtrees with the write operation that can be exploited to impact the network monitoring. An incorrect condition could cause frequent scaling operation to be executed causing harm to the network:

Further, following are the subtrees with the write operation that can be exploited by setting an incorrect grouping operation for the VN operation impacting the network monitoring:

Some of the readable data nodes in this YANG module may be considered sensitive or vulnerable in some network environments. It is thus important to control read access (e.g., via get, get-config, or notification) to these data nodes. These are the subtrees with the read operations that can be exploited to learn real-time (and sensitive) telemetry information about the TE tunnels and VN:

9. IANA Considerations

This document registers the following namespace URIs in the IETF XML registry [RFC3688]:

--------------------------------------------------------------------
URI: urn:ietf:params:xml:ns:yang:ietf-te-telemetry
Registrant Contact: The IESG.
XML: N/A, the requested URI is an XML namespace.
--------------------------------------------------------------------

--------------------------------------------------------------------
URI: urn:ietf:params:xml:ns:yang:ietf-vn-telemetry
Registrant Contact: The IESG.
XML: N/A, the requested URI is an XML namespace.
--------------------------------------------------------------------

This document registers the following YANG modules in the YANG Module registry.

Names registry [RFC7950]:

--------------------------------------------------------------------
name:         ietf-te-telemetry
namespace:    urn:ietf:params:xml:ns:yang:ietf-te-telemetry
prefix:       te-tel
reference:    RFC XXXX
--------------------------------------------------------------------

--------------------------------------------------------------------
name:         ietf-vn-telemetry
namespace:    urn:ietf:params:xml:ns:yang:ietf-vn-telemetry
prefix:       vn-tel
reference:    RFC XXXX
--------------------------------------------------------------------

10. Acknowledgements

We thank Adrian Farrel, Rakesh Gandhi, Tarek Saad, Igor Bryskin, Kenichi Ogaki, and Greg Mirsky for useful discussions and their suggestions for this work.

11. References

11.1. Normative References

[I-D.ietf-teas-actn-vn-yang]
Lee, Y., Dhody, D., Ceccarelli, D., Bryskin, I., and B. Y. Yoon, "A YANG Data Model for VN Operation", Work in Progress, Internet-Draft, draft-ietf-teas-actn-vn-yang-14, , <https://datatracker.ietf.org/doc/html/draft-ietf-teas-actn-vn-yang-14>.
[I-D.ietf-teas-yang-te]
Saad, T., Gandhi, R., Liu, X., Beeram, V. P., Bryskin, I., and O. G. D. Dios, "A YANG Data Model for Traffic Engineering Tunnels, Label Switched Paths and Interfaces", Work in Progress, Internet-Draft, draft-ietf-teas-yang-te-29, , <https://datatracker.ietf.org/doc/html/draft-ietf-teas-yang-te-29>.
[RFC3688]
Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688, DOI 10.17487/RFC3688, , <https://www.rfc-editor.org/info/rfc3688>.
[RFC6241]
Enns, R., Ed., Bjorklund, M., Ed., Schoenwaelder, J., Ed., and A. Bierman, Ed., "Network Configuration Protocol (NETCONF)", RFC 6241, DOI 10.17487/RFC6241, , <https://www.rfc-editor.org/info/rfc6241>.
[RFC6242]
Wasserman, M., "Using the NETCONF Protocol over Secure Shell (SSH)", RFC 6242, DOI 10.17487/RFC6242, , <https://www.rfc-editor.org/info/rfc6242>.
[RFC7926]
Farrel, A., Ed., Drake, J., Bitar, N., Swallow, G., Ceccarelli, D., and X. Zhang, "Problem Statement and Architecture for Information Exchange between Interconnected Traffic-Engineered Networks", BCP 206, RFC 7926, DOI 10.17487/RFC7926, , <https://www.rfc-editor.org/info/rfc7926>.
[RFC7950]
Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, , <https://www.rfc-editor.org/info/rfc7950>.
[RFC8040]
Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF Protocol", RFC 8040, DOI 10.17487/RFC8040, , <https://www.rfc-editor.org/info/rfc8040>.
[RFC8233]
Dhody, D., Wu, Q., Manral, V., Ali, Z., and K. Kumaki, "Extensions to the Path Computation Element Communication Protocol (PCEP) to Compute Service-Aware Label Switched Paths (LSPs)", RFC 8233, DOI 10.17487/RFC8233, , <https://www.rfc-editor.org/info/rfc8233>.
[RFC8340]
Bjorklund, M. and L. Berger, Ed., "YANG Tree Diagrams", BCP 215, RFC 8340, DOI 10.17487/RFC8340, , <https://www.rfc-editor.org/info/rfc8340>.
[RFC8341]
Bierman, A. and M. Bjorklund, "Network Configuration Access Control Model", STD 91, RFC 8341, DOI 10.17487/RFC8341, , <https://www.rfc-editor.org/info/rfc8341>.
[RFC8342]
Bjorklund, M., Schoenwaelder, J., Shafer, P., Watsen, K., and R. Wilton, "Network Management Datastore Architecture (NMDA)", RFC 8342, DOI 10.17487/RFC8342, , <https://www.rfc-editor.org/info/rfc8342>.
[RFC8446]
Rescorla, E., "The Transport Layer Security (TLS) Protocol Version 1.3", RFC 8446, DOI 10.17487/RFC8446, , <https://www.rfc-editor.org/info/rfc8446>.
[RFC8640]
Voit, E., Clemm, A., Gonzalez Prieto, A., Nilsen-Nygaard, E., and A. Tripathy, "Dynamic Subscription to YANG Events and Datastores over NETCONF", RFC 8640, DOI 10.17487/RFC8640, , <https://www.rfc-editor.org/info/rfc8640>.
[RFC8641]
Clemm, A. and E. Voit, "Subscription to YANG Notifications for Datastore Updates", RFC 8641, DOI 10.17487/RFC8641, , <https://www.rfc-editor.org/info/rfc8641>.
[RFC8776]
Saad, T., Gandhi, R., Liu, X., Beeram, V., and I. Bryskin, "Common YANG Data Types for Traffic Engineering", RFC 8776, DOI 10.17487/RFC8776, , <https://www.rfc-editor.org/info/rfc8776>.

11.2. Informative References

[I-D.ietf-opsawg-yang-vpn-service-pm]
Wu, B., Wu, Q., Boucadair, M., Dios, O. G. D., and B. Wen, "A YANG Model for Network and VPN Service Performance Monitoring", Work in Progress, Internet-Draft, draft-ietf-opsawg-yang-vpn-service-pm-03, , <https://datatracker.ietf.org/doc/html/draft-ietf-opsawg-yang-vpn-service-pm-03>.
[RFC7471]
Giacalone, S., Ward, D., Drake, J., Atlas, A., and S. Previdi, "OSPF Traffic Engineering (TE) Metric Extensions", RFC 7471, DOI 10.17487/RFC7471, , <https://www.rfc-editor.org/info/rfc7471>.
[RFC7823]
Atlas, A., Drake, J., Giacalone, S., and S. Previdi, "Performance-Based Path Selection for Explicitly Routed Label Switched Paths (LSPs) Using TE Metric Extensions", RFC 7823, DOI 10.17487/RFC7823, , <https://www.rfc-editor.org/info/rfc7823>.
[RFC8309]
Wu, Q., Liu, W., and A. Farrel, "Service Models Explained", RFC 8309, DOI 10.17487/RFC8309, , <https://www.rfc-editor.org/info/rfc8309>.
[RFC8453]
Ceccarelli, D., Ed. and Y. Lee, Ed., "Framework for Abstraction and Control of TE Networks (ACTN)", RFC 8453, DOI 10.17487/RFC8453, , <https://www.rfc-editor.org/info/rfc8453>.
[RFC8570]
Ginsberg, L., Ed., Previdi, S., Ed., Giacalone, S., Ward, D., Drake, J., and Q. Wu, "IS-IS Traffic Engineering (TE) Metric Extensions", RFC 8570, DOI 10.17487/RFC8570, , <https://www.rfc-editor.org/info/rfc8570>.

Appendix A. Out of Scope

This document exclusively focus on performance monitoring telemetry and scaling intent mechanisms of the underlying transport (TE-tunnels and Virtual Networks (VNs)). The performance monitoring of the services is out of scope. See Section 3.3 for details about VPN performance monitoring. Similarly performance monitoring of IETF network slices could be developed and it is clearly out of scope of this document.

Authors' Addresses

Young Lee (editor)
Samsung Electronics
Dhruv Dhody (editor)
Huawei Technologies
Divyashree Techno Park, Whitefield
Bangalore 560066
Karnataka
India
Satish Karunanithi
Huawei Technologies
Divyashree Techno Park, Whitefield
Bangalore 560066
Karnataka
India
Ricard Vilalta
CTTC
Centre Tecnologic de Telecomunicacions de Catalunya (CTTC/CERCA)
Barcelona
Spain
Daniel King
Lancaster University
Daniele Ceccarelli
Ericsson
Torshamnsgatan,48
Stockholm, Sweden