Guidelines for Autonomic Service Agents

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 28 December 2021.¶

Copyright Notice

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶

Table of Contents

1. Introduction

This document proposes guidelines for the design of Autonomic Service Agents (ASAs) in the context of an Autonomic Network (AN) based on the Autonomic Network Infrastructure (ANI) outlined in the ANIMA reference model [RFC8993]. This infrastructure makes use of the Autonomic Control Plane (ACP) [RFC8994] and the Generic Autonomic Signaling Protocol (GRASP) [RFC8990]. This document is a contribution to the description of an autonomic networks ecosystem, recognizing that a deployable autonomic network needs more than just ACP and GRASP implementations. Such an autonomic network must achieve management tasks that a Network Operations Center (NOC) cannot readily achieve manually, such as continuous resource optimization or automated fault detection and repair. These tasks, and other management automation goals, are described at length in [RFC7575]. The net result should be significant improvement of operational metrics. To achieve this objective, the autonomic networks ecosystem must include at least a library of ASAs and corresponding GRASP objective definitions. There must also be tools to deploy and oversee ASAs, and integration with existing operational mechanisms [RFC8368]. However, this document focuses on the design of ASAs, with some reference to implementation and operational aspects.¶

There is a considerable literature about autonomic agents with a variety of proposals about how they should be characterized. Some examples are [DeMola06], [Huebscher08], [Movahedi12] and [GANA13]. However, for the present document, the basic definitions and goals for autonomic networking given in [RFC7575] apply . According to RFC 7575, an Autonomic Service Agent is "An agent implemented on an autonomic node that implements an autonomic function, either in part (in the case of a distributed function) or whole."¶

ASAs must be distinguished from other forms of software component. They are components of network or service management; they do not in themselves provide services to end users. They do however provide management services to network operators and administrators. For example, the services envisaged for network function virtualisation [RFC8568] or for service function chaining [RFC7665] might be managed by an ASA rather than by traditional configuration tools.¶

Another example is that an existing script for locally monitoring or configuring functions or services on a router could be upgraded as an ASA that could communicate with peer scripts on neighboring or remote routers. A high-level API will allow such upgraded scripts to take full advantage of the secure ACP and the discovery, negotiation and synchronization features of GRASP. Familiar tasks such as configuring an Interior Gateway Protocol (IGP) on neighboring routers or even exchanging IGP security keys could be performed securely in this way. This document mainly addresses issues affecting quite complex ASAs, but the most useful ones may in fact be rather simple developments from existing scripts.¶

The reference model [RFC8993] for autonomic networks explains further the functionality of ASAs by adding "[An ASA is] a process that makes use of the features provided by the ANI to achieve its own goals, usually including interaction with other ASAs via the GRASP protocol [RFC8990] or otherwise. Of course it also interacts with the specific targets of its function, using any suitable mechanism. Unless its function is very simple, the ASA will need to handle overlapping asynchronous operations. It may therefore be a quite complex piece of software in its own right, forming part of the application layer above the ANI."¶

As mentioned, there will certainly be simple ASAs that manage a single objective in a straightforward way and do not need asynchronous operations. In such a case, many aspects of the current document do not apply. However, in the general case, an ASA may be a relatively complex software component that will in many cases control and monitor simpler entities in the same or remote host(s). For example, a device controller that manages tens or hundreds of simple devices might contain a single ASA.¶

The remainder of this document offers guidance on the design of such ASAs.¶

2. Logical Structure of an Autonomic Service Agent

As mentioned above, all but the simplest ASAs will need to suport asynchronous operations. Not all programming environments explicitly support multi-threading. In that case, an 'event loop' style of implementation could be adopted, in which case each thread would be implemented as an event handler called in turn by the main loop. For this, the GRASP API (Section 3.3) must provide non-blocking calls and possibly support callbacks. When necessary, the GRASP session identifier will be used to distinguish simultaneous operations.¶

A typical ASA will have a main thread that performs various initial housekeeping actions such as:¶

• Obtain authorization credentials, if needed.¶
• Register the ASA with GRASP.¶
• Acquire relevant policy parameters.¶
• Define data structures for relevant GRASP objectives.¶
• Register with GRASP those objectives that it will actively manage.¶
• Launch a self-monitoring thread.¶
• Enter its main loop.¶

The logic of the main loop will depend on the details of the autonomic function concerned. Whenever asynchronous operations are required, extra threads will be launched, or events added to the event loop. Examples include:¶

• Repeatedly flood an objective to the AN, so that any ASA can receive the objective's latest value.¶
• Accept incoming synchronization requests for an objective managed by this ASA.¶
• Accept incoming negotiation requests for an objective managed by this ASA, and then conduct the resulting negotiation with the counterpart ASA.¶
• Manage subsidiary non-autonomic devices directly.¶

These threads or events should all either exit after their job is done, or enter a wait state for new work, to avoid blocking others unnecessarily.¶

According to the degree of parallelism needed by the application, some of these threads or events might be launched in multiple instances. In particular, if negotiation sessions with other ASAs are expected to be long or to involve wait states, the ASA designer might allow for multiple simultaneous negotiating threads, with appropriate use of queues and locks to maintain consistency.¶

The main loop itself could act as the initiator of synchronization requests or negotiation requests, when the ASA needs data or resources from other ASAs. In particular, the main loop should watch for changes in policy parameters that affect its operation. It should also do whatever is required to avoid unnecessary resource consumption, such as including an arbitrary wait time in each cycle of the main loop.¶

The self-monitoring thread is of considerable importance. Autonomic service agents must never fail. To a large extent this depends on careful coding and testing, with no unhandled error returns or exceptions, but if there is nevertheless some sort of failure, the self-monitoring thread should detect it, fix it if possible, and in the worst case restart the entire ASA.¶

Appendix B presents some example logic flows in informal pseudocode.¶

3. Interaction with the Autonomic Networking Infrastructure

4. Interaction with Non-Autonomic Components

An ASA, to have any external effects, must also interact with non-autonomic components of the node where it is installed. For example, an ASA whose purpose is to manage a resource must interact with that resource. An ASA whose purpose is to manage an entity that is already managed by local software must interact with that software. For example, if such management is performed by NETCONF [RFC6241], the ASA must interact directly with the NETCONF server in the same node.¶

In an environment where systems are virtualized and specialized using techniques such as network function virtualization or network slicing, there will be a design choice whether ASAs are deployed once per physical node or once per virtual context. A related issue is whether the ANI as a whole is deployed once on a physical network, or whether several virtual ANIs are deployed. This aspect needs to be considered by the ASA designer.¶

5. Design of GRASP Objectives

The general rules for the format of GRASP Objective options, their names, and IANA registration are given in [RFC8990]. Additionally that document discusses various general considerations for the design of objectives, which are not repeated here. However, note that the GRASP protocol, like HTTP, does not provide transactional integrity. In particular, steps in a GRASP negotiation are not idempotent. The design of a GRASP objective and the logic flow of the ASA should take this into account. For example, if an ASA is allocating part of a shared resource to other ASAs, it needs to ensure that the same part of the resource is not allocated twice. The easiest way is to run only one negotiation at a time. If an ASA is capable of overlapping several negotiations, it must avoid interference between these negotiations.¶

Negotiations will always end, normally because one end or the other declares success or failure. If this does not happen, either a timeout or exhaustion of the loop count will occur. The definition of a GRASP objective should describe a specific negotiation policy if it is not self-evident.¶

GRASP allows a 'dry run' mode of negotiation, where a negotiation session follows its normal course but is not committed at either end until a subsequent live negotiation session. If 'dry run' mode is defined for the objective, its specification, and every implementation, must consider what state needs to be saved following a dry run negotiation, such that a subsequent live negotiation can be expected to succeed. It must be clear how long this state is kept, and what happens if the live negotiation occurs after this state is deleted. An ASA that requests a dry run negotiation must take account of the possibility that a successful dry run is followed by a failed live negotiation. Because of these complexities, the dry run mechanism should only be supported by objectives and ASAs where there is a significant benefit from it.¶

The actual value field of an objective is limited by the GRASP protocol definition to any data structure that can be expressed in Concise Binary Object Representation (CBOR) [RFC7049]. For some objectives, a single data item will suffice; for example an integer, a floating point number or a UTF-8 string. For more complex cases, a simple tuple structure such as [item1, item2, item3] could be used. Since CBOR is closely linked to JSON, it is also rather easy to define an objective whose value is a JSON structure. The formats acceptable by the GRASP API will limit the options in practice. A generic solution is for the API to accept and deliver the value field in raw CBOR, with the ASA itself encoding and decoding it via a CBOR library.¶

A mapping from YANG to CBOR is defined by [I-D.ietf-core-yang-cbor]. Subject to the size limit defined for GRASP messages, nothing prevents objectives using YANG in this way.¶

6. Life Cycle

In simple cases, Autonomic functions could be permanent, in the sense that ASAs are shipped as part of a product and persist throughout the product's life. However, in complex cases, a more likely situation is that ASAs need to be installed or updated dynamically, because of new requirements or bugs. This section describes one approach to the resulting life cycle.¶

Because continuity of service is fundamental to autonomic networking, the process of seamlessly replacing a running instance of an ASA with a new version needs to be part of the ASA's design. The implication of service continuity on the design of ASAs can be illustrated along the three main phases of the ASA life-cycle, namely Installation, Instantiation and Operation.¶

                  +--------------+
Undeployed ------>|              |------> Undeployed
                  |  Installed   |
              +-->|              |---+
     Mandate  |   +--------------+   | Receives a
   is revoked |   +--------------+   |  Mandate
              +---|              |<--+
                  | Instantiated |
              +-->|              |---+
          set |   +--------------+   | set
         down |   +--------------+   | up
              +---|              |<--+
                  |  Operational |
                  |              |
                  +--------------+

7. Coordination between Autonomic Functions

Some autonomic functions will be completely independent of each other. However, others are at risk of interfering with each other - for example, two different optimization functions might both attempt to modify the same underlying parameter in different ways. In a complete system, a method is needed of identifying ASAs that might interfere with each other and coordinating their actions when necessary. This issue is considered in "Autonomic Functions Coordination" [I-D.ciavaglia-anima-coordination].¶

8. Coordination with Traditional Management Functions

Some ASAs will have functions that overlap with existing configuration tools and network management mechanisms such as command line interfaces, DHCP, DHCPv6, SNMP, NETCONF, and RESTCONF. This is of course an existing problem whenever multiple configuration tools are in use by the NOC. Each ASA designer will need to consider this issue and how to avoid clashes and inconsistencies. Some specific considerations for interaction with OAM tools are given in [RFC8368]. As another example, [RFC8992] describes how autonomic management of IPv6 prefixes can interact with prefix delegation via DHCPv6. The description of a GRASP objective and of an ASA using it should include a discussion of any such interactions.¶

9. Data Models

Management functions often include a data model, quite likely to be expressed in a formal notation such as YANG. This aspect should not be an afterthought in the design of an ASA. To the contrary, the design of the ASA and of its GRASP objectives should match the data model; as noted in Section 5, YANG serialized as CBOR may be used directly as the value of a GRASP objective.¶

10. Robustness

It is of great importance that all components of an autonomic system are highly robust. In principle they must never fail. This section lists various aspects of robustness that ASA designers should consider.¶

1. If despite all precautions, an ASA does encounter a fatal error, it should in any case restart automatically and try again. To mitigate a hard loop in case of persistent failure, a suitable pause should be inserted before such a restart. The length of the pause depends on the use case.¶
2. If a newly received or calculated value for a parameter falls out of bounds, the corresponding parameter should be either left unchanged or restored to a safe value.¶
3. If a GRASP synchronization or negotiation session fails for any reason, it may be repeated after a suitable pause. The length of the pause depends on the use case.¶
4. If a session fails repeatedly, the ASA should consider that its peer has failed, and cause GRASP to flush its discovery cache and repeat peer discovery.¶
5. In any case, it may be prudent to repeat discovery periodically, depending on the use case.¶
6. Any received GRASP message should be checked. If it is wrongly formatted, it should be ignored. Within a unicast session, an Invalid message (M_INVALID) may be sent. This function may be provided by the GRASP implementation itself.¶
7. Any received GRASP objective should be checked. Basic formatting errors like invalid CBOR will likely be detected by GRASP itself, but the ASA is responsible for checking the precise syntax and semantics of a received objective. If it is wrongly formatted, it should be ignored. Within a negotiation session, a Negotiation End message (M_END) with a Decline option (O_DECLINE) should be sent. An ASA may log such events for diagnostic purposes.¶
8. On the other hand, the definitions of GRASP objectives are very likely to be extended, using the flexibility of CBOR or JSON. Therefore, ASAs should be able to deal gracefully with unknown components within the values of objectives.¶
9. If an ASA receives either an Invalid message (M_INVALID) or a Negotiation End message (M_END) with a Decline option (O_DECLINE), one possible reason is that the peer ASA does not support a new feature of either GRASP or of the objective in question. In such a case the ASA may choose to repeat the operation concerned without using that new feature.¶
10. All other possible exceptions should be handled in an orderly way. There should be no such thing as an unhandled exception (but see point 1 above).¶

At a slightly more general level, ASAs are not services in themselves, but they automate services. This has a fundamental impact on how to design robust ASAs. In general, when an ASA observes a particular state [1] of operations of the services/resources it controls, it typically aims to improve this state to a better state, say [2]. Ideally, the ASA is built so that it can ensure that any error encountered can still lead to returning to [1] instead of a state [3] which is worse than [1]. One example instance of this principle is "make-before-break" used in reconfiguration of routing protocols in manual operations. This principle of operations can accordingly be coded into the operation of an ASA. The GRASP dry run option mentioned in Section 5 is another tool helpful for this ASA design goal of "test-before-make".¶

11. Security Considerations

ASAs are intended to run in an environment that is protected by the Autonomic Control Plane [RFC8994], admission to which depends on an initial secure bootstrap process such as [RFC8995]. Such an ACP can provide keying material for mutual authentication between ASAs as well as confidential communication channels for messages between ASAs. In some deployments, a secure partition of the link layer might be used instead. However, this does not relieve ASAs of responsibility for security. When ASAs configure or manage network elements outside the ACP, potentially in a different physical node, they must interact with other non-autonomic software components to perform their management functions. The details are specific to each case, but this has an important security implication. An ASA might act as a loophole by which the managed entity could penetrate the security boundary of the ANI. Thus, ASAs must be designed to avoid such loopholes, and should if possible operate in an unprivileged mode. In particular, they must use secure techniques and carefully validate any incoming information. This will apply in particular when an ASA interacts with a management component such as a NETCONF server.¶

As appropriate to their specific functions, ASAs should take account of relevant privacy considerations [RFC6973].¶

The initial version of the autonomic infrastructure assumes that all autonomic nodes are trusted by virtue of their admission to the ACP. ASAs are therefore trusted to manipulate any GRASP objective, simply because they are installed on a node that has successfully joined the ACP. In the general case, a node may have multiple roles and a role may use multiple ASAs, each using multiple GRASP objectives. Additional mechanisms for the fine-grained authorization of nodes and ASAs to manipulate specific GRASP objectives could be designed. Independently of this, interfaces between ASAs and the router configuration/monitoring services of the node can be subject to authentication that provides more fine grained authorization for specific services. These additional authentication parameters could be passed to an ASA during its instantiation phase.¶

12. IANA Considerations

This document makes no request of the IANA.¶

13. Acknowledgements

Useful comments were received from Michael Behringer Toerless Eckert, Alex Galis, Bing Liu, Michael Richardson, and other members of the ANIMA WG.¶

Appendix A. Change log

This section is to be removed before publishing as an RFC.¶

draft-ietf-anima-asa-guidelines-01, 2021-06-27:¶

• Incorporated shepherd's review comments¶
• Editorial fixes¶

draft-ietf-anima-asa-guidelines-00, 2020-11-14:¶

• Adopted by WG¶
• Editorial fixes¶

draft-carpenter-anima-asa-guidelines-09, 2020-07-25:¶

• Additional text on future authorization.¶
• Editorial fixes¶

draft-carpenter-anima-asa-guidelines-08, 2020-01-10:¶

• Introduced notion of autonomic ecosystem.¶
• Minor technical clarifications.¶
• Converted to v3 format.¶

draft-carpenter-anima-asa-guidelines-07, 2019-07-17:¶

• Improved explanation of threading vs event-loop¶
• Other editorial improvements.¶

draft-carpenter-anima-asa-guidelines-06, 2018-01-07:¶

• Expanded and improved example logic flow.¶
• Editorial corrections.¶

draft-carpenter-anima-asa-guidelines-05, 2018-06-30:¶

• Added section on relationshp with non-autonomic components.¶
• Editorial corrections.¶

draft-carpenter-anima-asa-guidelines-04, 2018-03-03:¶

• Added note about simple ASAs.¶
• Added note about NFV/SFC services.¶
• Improved text about threading v event loop model¶
• Added section about coordination with traditional tools.¶
• Added appendix with example logic flow.¶

draft-carpenter-anima-asa-guidelines-03, 2017-10-25:¶

• Added details on life cycle.¶
• Added details on robustness.¶
• Added co-authors.¶

draft-carpenter-anima-asa-guidelines-02, 2017-07-01:¶

• Expanded description of event-loop case.¶
• Added note about 'dry run' mode.¶

draft-carpenter-anima-asa-guidelines-01, 2017-01-06:¶

• More sections filled in.¶

draft-carpenter-anima-asa-guidelines-00, 2016-09-30:¶

• Initial version¶

Appendix B. Example Logic Flows

This appendix describes generic logic flows for an Autonomic Service Agent (ASA) for resource management. Note that these are illustrative examples, and in no sense requirements. As long as the rules of GRASP are followed, a real implementation could be different. The reader is assumed to be familiar with GRASP [RFC8990] and its conceptual API [RFC8991].¶

A complete autonomic function for a resource would consist of a number of instances of the ASA placed at relevant points in a network. Specific details will of course depend on the resource concerned. One example is IP address prefix management, as specified in [RFC8992]. In this case, an instance of the ASA would exist in each delegating router.¶

An underlying assumption is that there is an initial source of the resource in question, referred to here as an origin ASA. The other ASAs, known as delegators, obtain supplies of the resource from the origin, and then delegate quantities of the resource to consumers that request it, and recover it when no longer needed.¶

Another assumption is there is a set of network wide policy parameters, which the origin will provide to the delegators. These parameters will control how the delegators decide how much resource to provide to consumers. Thus the ASA logic has two operating modes: origin and delegator. When running as an origin, it starts by obtaining a quantity of the resource from the NOC, and it acts as a source of policy parameters, via both GRASP flooding and GRASP synchronization. (In some scenarios, flooding or synchronization alone might be sufficient, but this example includes both.)¶

When running as a delegator, it starts with an empty resource pool, it acquires the policy parameters by GRASP synchronization, and it delegates quantities of the resource to consumers that request it. Both as an origin and as a delegator, when its pool is low it seeks quantities of the resource by requesting GRASP negotiation with peer ASAs. When its pool is sufficient, it hands out resource to peer ASAs in response to negotiation requests. Thus, over time, the initial resource pool held by the origin will be shared among all the delegators according to demand.¶

In theory a network could include any number of origins and any number of delegators, with the only condition being that each origin's initial resource pool is unique. A realistic scenario is to have exactly one origin and as many delegators as you like. A scenario with no origin is useless.¶

An implementation requirement is that resource pools are kept in stable storage. Otherwise, if a delegator exits for any reason, all the resources it has obtained or delegated are lost. If an origin exits, its entire spare pool is lost. The logic for using stable storage and for crash recovery is not included in the pseudocode below.¶

The description below does not implement GRASP's 'dry run' function. That would require temporarily marking any resource handed out in a dry run negotiation as reserved, until either the peer obtains it in a live run, or a suitable timeout expires.¶

The main data structures used in each instance of the ASA are:¶

• The resource_pool, for example an ordered list of available resources. Depending on the nature of the resource, units of resource are split when appropriate, and a background garbage collector recombines split resources if they are returned to the pool.¶
• The delegated_list, where a delegator stores the resources it has given to consumers routers.¶

Possible main logic flows are below, using a threaded implementation model. The transformation to an event loop model should be apparent - each thread would correspond to one event in the event loop.¶

The GRASP objectives are as follows:¶

• ["EX1.Resource", flags, loop_count, value] where the value depends on the resource concerned, but will typically include its size and identification.¶
• ["EX1.Params", flags, loop_count, value] where the value will be, for example, a JSON object defining the applicable parameters.¶

In the outline logic flows below, these objectives are represented simply by their names.¶

<CODE BEGINS>

MAIN PROGRAM:

Create empty resource_pool (and an associated lock)
Create empty delegated_list
Determine whether to act as origin
if origin:
    Obtain initial resource_pool contents from NOC
    Obtain value of EX1.Params from NOC
Register ASA with GRASP
Register GRASP objectives EX1.Resource and EX1.Params
if origin:
    Start FLOODER thread to flood EX1.Params
    Start SYNCHRONIZER listener for EX1.Params
Start MAIN_NEGOTIATOR thread for EX1.Resource
if not origin:
    Obtain value of EX1.Params from GRASP flood or synchronization
    Start DELEGATOR thread
Start GARBAGE_COLLECTOR thread
do forever:
    good_peer = none
    if resource_pool is low:
        Calculate amount A of resource needed
        Discover peers using GRASP M_DISCOVER / M_RESPONSE
        if good_peer in peers:
            peer = good_peer
        else:
            peer =  #any choice among peers
            grasp.request_negotiate("EX1.Resource", peer)
            i.e., send M_REQ_NEG
            Wait for response (M_NEGOTIATE, M_END or M_WAIT)
            if OK:
                if offered amount of resource sufficient:
                    Send M_END + O_ACCEPT #negotiation succeeded
                    Add resource to pool
                    good_peer = peer
                else:
                    Send M_END + O_DECLINE #negotiation failed
    sleep() #sleep time depends on application scenario

MAIN_NEGOTIATOR thread:

do forever:
    grasp.listen_negotiate("EX1.Resource")
    i.e., wait for M_REQ_NEG
    Start a separate new NEGOTIATOR thread for requested amount A

NEGOTIATOR thread:

Request resource amount A from resource_pool
if not OK:
    while not OK and A > Amin:
        A = A-1
        Request resource amount A from resource_pool
if OK:
    Offer resource amount A to peer by GRASP M_NEGOTIATE
    if received M_END + O_ACCEPT:
        #negotiation succeeded
    elif received M_END + O_DECLINE or other error:
        #negotiation failed
else:
    Send M_END + O_DECLINE #negotiation failed

DELEGATOR thread:

do forever:
    Wait for request or release for resource amount A
    if request:
        Get resource amount A from resource_pool
        if OK:
            Delegate resource to consumer
            Record in delegated_list
        else:
            Signal failure to consumer
            Signal main thread that resource_pool is low
    else:
        Delete resource from delegated_list
        Return resource amount A to resource_pool

SYNCHRONIZER thread:

do forever:
    Wait for  M_REQ_SYN message for EX1.Params
    Reply with M_SYNCH message for EX1.Params

FLOODER thread:

do forever:
    Send M_FLOOD message for EX1.Params
    sleep() #sleep time depends on application scenario

GARBAGE_COLLECTOR thread:

do forever:
    Search resource_pool for adjacent resources
    Merge adjacent resources
    sleep() #sleep time depends on application scenario

<CODE ENDS>

Authors' Addresses