| Internet-Draft | Augmented-by Addition into the IETF-YANG | March 2024 |
| Lin, et al. | Expires 4 September 2024 | [Page] |
This document augments the ietf-yang-library in [RFC8525] to provide the augmented-by list. It facilitates the process of obtaining the entire dependencies of YANG model, by directly querying the server's YANG module.¶
This note is to be removed before publishing as an RFC.¶
Source for this draft and an issue tracker can be found at https://github.com/Zephyre777/draft-lincla-netconf-yang-library-augmentation.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 4 September 2024.¶
Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
The YANG library [RFC8525] specifies a YANG module that provides the information about the YANG models and datastores to facilitate a client application to fully utilize and understand the YANG data modelling language. To know the YANG dependencies, [RFC8525] has defined and provided the submodule list and the YANG modules deviation list. However, the YANG modules augmentation is not provided.¶
According to [RFC7950], both augmentations and deviations are defining contents external to the model, but applying internally for the model. It is important to know the augmentation and deviation as they are dependencies of the model, but it is also difficult because they are defined externally. When we try to use the ietf-yang-library in [RFC8525] to obtain the reverse dependencies (Augmentations and deviations), the augmentation is not defined in it.¶
However, the augmentation and the deviation work similarly as YANG modules dependency. Therefore, it is reasonable to document them the same way in the IETF YANG library. Besides, it will be easier to determine the reverse dependency if the augmentation is directly available, through a GET request into this new YANG model.¶
This draft augment the ietf-yang-library to include the YANG modules augmentation information.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The terminology from [RFC8525] is used in this document¶
Tree diagrams in this document use the notation defined in [RFC8340] .¶
When using one YANG model, it is important to make sure that all its dependencies are presented. In [RFC7950] there are four dependencies for one YANG mode:¶
The import and include are direct dependencies while the augmentation and deviation are reverse dependencies. To know a specific YANG model's direct dependencies, we can parse this YANG model as the dependencies are directly specified (import and include statements").¶
As for the reverse dependencies, since they are defined externally, we cannot parse the YANG model itself to get them. Among all the methods for getting reverse dependency, getting the ietf-yang-library content is one of the most convenient ways.¶
However, the ietf-yang-library only provides the deviation list, but not the augmentation. It is reasonable to update it to provide the augmentation information since both augmentation and deviation have similar way of working (both are applied to the original model but invisible to them). A noticeable difference between deviations and augmentations is that the deviations are required to understand the API contract between the client and the server. But with some use cases arise as the requirement of automate network management, the augmentation becomes essential information for understanding the network, too.¶
As the demand arises for YANG-based telemetry [RFC8641], there is a need for real-time knowledge of a specific YANG model's dependency list when a specific YANG-Push message is received.¶
The alternative, for a YANG-push receiver, to collect and store the entire module set for every single server who could be streaming data, is not always practical. See the following reasons:¶
With similar central idea, two use cases are introduced in this section. One target solving the dependencies problems in a Data Mesh Telemetry System while the other aims at building a data catalog which makes YANG model inforamtion easily accessible.¶
A network analytics architecture that integrates YANG-push and Kafka is proposed in 2022 and is continuously growing and gaining influence, refer to the draft: An Architecture for YANG-Push to Apache Kafka Integration [I-D.netana-nmop-yang-kafka-integration].¶
In this open-sourced project covering Support of Versioning in YANG Notifications Subscription [I-D.ietf-netconf-yang-notifications-versioning], Support of Network Observation Timestamping in YANG Notifications [I-D.netconf-tgraf-yang-push-observation-time], among others, the purpose is to provide adequate information in the YANG-Push notification so that when it is received, the model and its dependency can be parsed and found automatically from the vantage point. The architecture relies on the information of YANG model and their dependency to realize, as one of its main goals is to solve the problem of missing YANG semantics when data is received in Time Series Database in the end. To solve the problem, a schema registry is introduced to store YANG models and all their relationship (direct dependency and reverse dependency).¶
Currently, the method used for finding model's reverse dependency is get-all-schemas, that is to retrieve all YANG modules from the device to the client's disk, enabling the client to fully understand the YANG model relationship. This process is heavy because in real situation, each device may have few thousands or even more YANG module implemented. Besides, pressure is also introduced due to the need of parsing modules and find all the dependencies.¶
Considering the telemetry real-time aspects, this extra delay in processing the dependencies through get-all-schemas is not ideal.¶
The YANG model proposed in the draft can be used in this architecture to release the stress of get-all-schemas and bring some extra benefits.¶
By providing the augmentation information, it enables the collector to get the YANG reverse dependencies by simply sending one <get> query to ietf-yang-library. In this regard, the process of acquiring full dependency becomes real-time action.¶
Compared with get-all-schemas, it enables collector to fetch up-to-date data since the queries are sent at run time, while the old approach forces collector to always work with never updated data. On another hand, user do not bother waiting for ten minutes every time when starting collector to either get data updated or to obtain YANG relationship.¶
Finding the YANG models implemented by a network device is paramount for configuring and monitoring the status of a network. However, since the inception of YANG the network industry has experienced a tsunami of YANG models developed by SDOs, open-source communities, and network vendors. This heterogeneity of YANG models, that vary from one network device model to another, makes the management of a multi-vendor network a big challenge for operators. [Martinez-Casanueva2023]¶
In this regard, a data catalog provides a registry of the datasets exposed by remote data sources for consumers to discover data of interest. Besides the location of the dataset (i.e., the data source), the data catalog registers additional metadata such as the data model (or schema) followed in the dataset or even related terms defined in a business glossary.¶
Data catalog solutions typically implement collectors that ingest metadata from the data sources themselves and also external metadata sources. For example, a Kafka Schema Registry is a metadata source that provides metadata about the data models followed by some data stored in a Kafka topic.¶
In this sense, a YANG-enabled network device can be considered as another kind of data source, which the Data Catalog can pull metadata from. For instance, the data catalog can include a connector that fetches metadata about the YANG models implemented by the network device. Combining these metadata with other such as the business concept "interface", would enable data consumers to discover which datasets related to the concept "interface" are exposed by the network device.¶
Network devices that implement YANG Library expose metadata about which YANG modules are implemented, and which are only imported. However, what a data consumer needs at the end are the YANG models implemented by the device, hence, the combination of implemented YANG modules with other YANG modules that might deviate or augment the formers.¶
Coming back to the example of datasets related to the "interface" concept, say we have a network device that implements the ietf-interfaces module [RFC8343] and the ietf-ip module [RFC8344], where the latter augments the former. For a data catalog to collect these metadata, a connector would retrieve YANG Library data from the target device. However, the current version of YANG Library would not satisfy the use case as it would tell that the device implements both ietf-interfaces and ietf-ip modules, but will miss the augment dependency between them.¶
The current workaround to this limitation is to, in combination with the YANG library data, additionally fetch both YANG modules and process them to discover that there is an augment dependency. This adds extra burden on the connector, which is forced to combine multiple metadata collection mechanisms. This process could be softened by extending YANG Library to also capture augment dependencies, in a similar fashion to deviation dependencies.¶
This YANG module augments the ietf-yang-library module by adding the augmented-by list in the "yang-library/module-set". The name Augmented-by indicated the modules by which the current module is being augmented. Note that this module only augments the ietf-yang-library defined in [RFC8525]. At the time of writing this document, most router vendors support [RFC7895], a previous revision of the ietf-yang-library YANG module; The module that augments [RFC7895] is provided in the appendix B.¶
The following is the YANG tree diagram for model ietf-yang-library-augmentedby.¶
module: ietf-yang-library-augmentedby
augment /yanglib:yang-library/yanglib:module-set/yanglib:module:
+--ro augmented-by* -> ../../yanglib:module/name
¶
The following is the YANG tree diagram[RFC8340] for the ietf-yang-library with the augmentation defined in module ietf-yang-library-augmentedby, including the RPCs and notifications.¶
module: ietf-yang-library
+--ro yang-library
| +--ro module-set* [name]
| | +--ro name string
| | +--ro module* [name]
| | | +--ro name yang:yang-identifier
| | | +--ro revision? revision-identifier
| | | +--ro namespace inet:uri
| | | +--ro location* inet:uri
| | | +--ro submodule* [name]
| | | | +--ro name yang:yang-identifier
| | | | +--ro revision? revision-identifier
| | | | +--ro location* inet:uri
| | | +--ro feature* yang:yang-identifier
| | | +--ro deviation* -> ../../module/name
| | | +--ro yanglib-aug:augmented-by*
-> ../../yanglib:module/name
| | +--ro import-only-module* [name revision]
| | +--ro name yang:yang-identifier
| | +--ro revision union
| | +--ro namespace inet:uri
| | +--ro location* inet:uri
| | +--ro submodule* [name]
| | +--ro name yang:yang-identifier
| | +--ro revision? revision-identifier
| | +--ro location* inet:uri
| +--ro schema* [name]
| | +--ro name string
| | +--ro module-set* -> ../../module-set/name
| +--ro datastore* [name]
| | +--ro name ds:datastore-ref
| | +--ro schema -> ../../schema/name
| +--ro content-id string
x--ro modules-state
x--ro module-set-id string
x--ro module* [name revision]
x--ro name yang:yang-identifier
x--ro revision union
+--ro schema? inet:uri
x--ro namespace inet:uri
x--ro feature* yang:yang-identifier
x--ro deviation* [name revision]
| x--ro name yang:yang-identifier
| x--ro revision union
x--ro conformance-type enumeration
x--ro submodule* [name revision]
x--ro name yang:yang-identifier
x--ro revision union
+--ro schema? inet:uri
notifications:
+---n yang-library-update
| +--ro content-id -> /yang-library/content-id
x---n yang-library-change
x--ro module-set-id -> /modules-state/module-set-id
¶
The YANG module source code of ietf-yang-library-augmentedby in which augmentation to the ietf-yang-library of [RFC8525] is defined.¶
<CODE BEGINS> file "ietf-yang-library-augmentedby@2023-10-27.yang"
module ietf-yang-library-augmentedby {
yang-version 1.1;
namespace "urn:ietf:params:xml:ns:yang:ietf-yang-library-augmentedby";
prefix yanglib-aug;
import ietf-yang-library {
prefix yanglib;
reference
"RFC 8525: YANG library";
}
organization
"IETF NETCONF (Network Configuration) Working Group";
contact
"WG Web: <https://datatracker.ietf.org/wg/netconf/>
WG List: <mailto:netconf@ietf.org>
Author: Zhuoyao Lin
<mailto:zephyre888@gmail.com>
Benoit Claise
<mailto:benoit.claise@huawei.com>
IGNACIO DOMINGUEZ MARTINEZ-CASANUEVA
<matilto:ignacio.dominguezmartinez@telefonica.com>";
description
"This module augments the ietf-yang-library defined in
[RFC8525] to provide not only the deviation list, but also
the augmented-by list, in order to give sufficient
information about the YANG models reverse dependency. It
facilitates the process of obtaining the entire
dependencies of YANG model.
The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL',
'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED',
'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document
are to be interpreted as described in BCP 14 (RFC 2119)
(RFC 8174) when, and only when, they appear in all
capitals, as shown here.
Copyright (c) 2022 IETF Trust and the persons identified as
authors of the code. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, is permitted pursuant to, and subject
to the license terms contained in, the Revised BSD License
set forth in Section 4.c of the IETF Trust's Legal Provisions
Relating to IETF Documents
(https://trustee.ietf.org/license-info).
This version of this YANG module is part of RFC XXXX; see the
RFC itself for full legal notices. ";
revision 2023-10-27 {
description
"Added list augmented-by in yang-library/module-set/module to
make the module store the entire reverse dependency information
(augmented-by and deviation).";
reference
"RFC XXXX: Support of augmentedby in ietf-yang-library";
}
augment "/yanglib:yang-library/yanglib:module-set/yanglib:module" {
description
"Augment the augmented-by list from module info with the
module-augmented-by grouping" ;
leaf-list augmented-by {
type leafref {
path "../../yanglib:module/yanglib:name";
}
description
"Leaf-list of the augmentation used by this server to
modify the conformance of the module associated with
this entry. Note that the same module can be used for
augmented-by for multiple modules, so the same
entry MAY appear within multiple 'module' entries.
This reference MUST NOT (directly or indirectly)
refer to the module being augmented.
Robust clients may want to make sure that they handle a
situation where a module augments itself (directly or
indirectly) gracefully.";
}
}
}
<CODE ENDS>¶
Note to the RFC-Editor: Please remove this section before publishing.¶
Here is the github repository for the YANG source code of this draft: https://github.com/Zephyre777/draft-lincla-netconf-yang-library-augmentation.git.¶
For the demo, please refer to the demo folder in this repository. There is a netopeer2 instance running with updated version of libyang and sysrepo(links are listed in the demo instruction).¶
The list name has been updated from "augmentation" to "augmented-by", in order to represent the usage clearly.¶
The leafref has been changed from absolute path "/yanglib:yang-libraray/yanglib:module-set/yanglib:module/yanglib:name" to relative path "../../yanglib:module/yanglib:name". The YANG validation in the appendix A shows that this path can work as expected.¶
Section 5 Implementation and section 6 Changes has been added.¶
This document has no actions for IANA.¶
This section gives a few examples that the user can try themselves with yanglint. This is created to prove the syntax correctness.¶
The valid example should pass the validation while the invalid one will not, because the module has augmented a module in another module-set, which is illegal according to the YANG source code.¶
<CODE BEGINS> file "example_valid.xml"
<yang-library xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<content-id>1</content-id>
<module-set>
<name>ms1</name>
<module>
<name>module1</name>
<revision>2024-02-29</revision>
<namespace>urn:ietf:params:xml:ns:yang:module1</namespace>
<augmented-by
xmlns="urn:ietf:params:xml:ns:yang:
ietf-yang-library-augmentedby">module2</augmented-by>
<augmented-by
xmlns="urn:ietf:params:xml:ns:yang:
ietf-yang-library-augmentedby">module3</augmented-by>
</module>
<module>
<name>module2</name>
<revision>2024-02-29</revision>
<namespace>urn:ietf:params:xml:ns:yang:module2</namespace>
</module>
<module>
<name>module3</name>
<revision>2024-02-29</revision>
<namespace>urn:ietf:params:xml:ns:yang:module3</namespace>
</module>
</module-set>
</yang-library>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>0</module-set-id>
</modules-state>
<CODE ENDS>¶
<CODE BEGINS> file "example_invalid.xml"
<yang-library xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<content-id>1</content-id>
<module-set>
<name>ms1</name>
<module>
<name>module1</name>
<revision>2024-02-29</revision>
<namespace>urn:ietf:params:xml:ns:yang:module1</namespace>
<augmented-by
xmlns="urn:ietf:params:xml:ns:yang:
ietf-yang-library-augmentedby">module2</augmented-by>
<augmented-by
xmlns="urn:ietf:params:xml:ns:yang:
ietf-yang-library-augmentedby">module3</augmented-by>
</module>
<module>
<name>module2</name>
<revision>2024-02-29</revision>
<namespace>urn:ietf:params:xml:ns:yang:module2</namespace>
</module>
<module>
<name>module3</name>
<revision>2024-02-29</revision>
<namespace>urn:ietf:params:xml:ns:yang:module3</namespace>
</module>
</module-set>
</yang-library>
<modules-state xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-library">
<module-set-id>0</module-set-id>
</modules-state>
<CODE ENDS>¶
This section defines the ietf-yang-library-rfc7895-augmentedby that augments the ietf-yang-library defined in [RFC7895]. The module-state/module list of this YANG module version is also defined in the [RFC8525] version though deprecated.¶
The following is the YANG tree diagram for ietf-yang-library-rfc7895-augmentedby augmenting RFC7895.¶
module: ietf-yang-library-rfc7895-augmentedby
augment /yanglib:modules-state/yanglib:module:
x--ro augmentedby* [name revision]
+--ro name -> /yanglib:modules-state/module/name
+--ro revision -> /yanglib:modules-state/module/revision
¶
The following is the full YANG tree diagram of ietf-yang-library-rfc7895-augmentedby augmenting ietf-yang-library defined in RFC7895.¶
module: ietf-yang-library
+--ro modules-state
+--ro module-set-id string
+--ro module* [name revision]
+--ro name yang:yang-identifier
+--ro revision union
+--ro schema? inet:uri
+--ro namespace inet:uri
+--ro feature* yang:yang-identifier
+--ro deviation* [name revision]
| +--ro name yang:yang-identifier
| +--ro revision union
+--ro conformance-type enumeration
+--ro submodule* [name revision]
| +--ro name yang:yang-identifier
| +--ro revision union
| +--ro schema? inet:uri
x--ro yanglib-aug:augmented-by* [name revision]
+--ro yanglib-aug:name
-> /yanglib:modules-state/module/name
+--ro yanglib-aug:revision
-> /yanglib:modules-state/module/revision
notifications:
+---n yang-library-change
+--ro module-set-id -> /modules-state/module-set-id
¶
The YANG module that augments the ietf-yang-library RFC7895.¶
<CODE BEGINS> file "ietf-yang-library-rfc7895-augmentedby@2023-10-27.yang"
module ietf-yang-library-rfc7895-augmentedby {
yang-version 1.1;
namespace "urn:ietf:params:xml:ns:yang:ietf-yang-library-rfc7895-augmentedby";
prefix yanglib-aug;
import ietf-yang-library {
prefix yanglib;
revision-date 2016-06-21;
reference
"RFC 7895: YANG Module Library.";
}
organization
"IETF NETCONF (Network Configuration) Working Group";
contact
"WG Web: <https://datatracker.ietf.org/wg/netconf/>
WG List: <mailto:netconf@ietf.org>
Author: Zhuoyao Lin
<mailto:zephyre888@gmail.com>
Author: Benoit Claise
<mailto:benoit.claise@huawei.com>
Author: IGNACIO DOMINGUEZ MARTINEZ-CASANUEVA
<matilto:ignacio.dominguezmartinez@telefonica.com>";
description
"This module augments the ietf-yang-library defined in [RFC7895]
to provide not only the deviation list, but also the
augmentedby list, in order to give sufficient information
about the YANG models reverse dependency. It facilitates
the process of obtaining the entire dependencies of YANG model.
The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL',
'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED',
'NOT RECOMMENDED', 'MAY', and 'OPTIONAL' in this document
are to be interpreted as described in BCP 14 (RFC 2119)
(RFC 8174) when, and only when, they appear in all
capitals, as shown here.
Copyright (c) 2022 IETF Trust and the persons identified as
authors of the code. All rights reserved.
Redistribution and use in source and binary forms, with or
without modification, is permitted pursuant to, and subject
to the license terms contained in, the Revised BSD License
set forth in Section 4.c of the IETF Trust's Legal Provisions
Relating to IETF Documents
(https://trustee.ietf.org/license-info).
This version of this YANG module is part of RFC XXXX; see the
RFC itself for full legal notices. ";
revision 2023-10-27 {
description
"Added list augmentedby in yang-library/modules-state/module to
make the module store the entire reverse dependency information
(augmentedby and deviation).";
reference
"RFC XXXX: Support of augmentedby in ietf-yang-library
defined in RFC7895";
}
augment "/yanglib:modules-state/yanglib:module" {
description
"Augment the augmentedby from module info with the
module-augmented-by grouping" ;
uses yanglib-aug:module-state-augmented-by;
}
/*
* Groupings
*/
grouping module-state-augmented-by {
description
"This grouping defines a list with keys being the module
name and revison. The list contains the augmented-by list.";
list augmented-by {
key "name revision";
status deprecated;
description
"List of YANG augmented-by module names and revisions
used by this server to modify the conformance of
the module associated with this entry. Note that
the same module can be used for augmented-by for
multiple modules, so the same entry MAY appear
within multiple 'module' entries.
The augment module MUST be present in the 'module'
list, with the same name and revision values.
The 'conformance-type' value will be 'implement' for
the augment module.";
leaf name {
type leafref {
path "/yanglib:modules-state/yanglib:module/yanglib:name";
}
description
"Identifies a given module in the yang library by
its name.";
}
leaf revision {
type leafref {
path "/yanglib:modules-state/yanglib:module/yanglib:revision";
}
description
"Revision of the module";
}
}
}
}
<CODE ENDS>¶
The following people all contributed to creating this document:¶
The author would like to thanks Jan Lindblad and Jean Quilbeuf for his help during the design of the YANG module.¶