| Internet-Draft | iiot-framework | October 2021 |
| Makhijani & Dong | Expires 28 April 2022 | [Page] |
Industry control networks host a diverse set of non-internet protocols supporting Industrial-IoT and legacy device connections. The integration between traditional information technology (IT) and operational technology (OT) so far has centered around collection of real-time data from devices in OT environment for consumption within the enterprise IT networks. However, improvements in process control and automation require a far better interworking between the OT and IT applications. This document provides a reference framework for integrated industry networks (IIN). It highlights interfaces and their characteristics required for interconnecting components of OT and IT that maybe moved to the cloud or edges. It suggests the use of IIN to bridge the differences between OT and IETF technologies.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 28 April 2022.¶
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶
There is very little cross-over between the network technologies used in the OT and IT environment. The OT networks are responsible for automation and process control on premises such as factory floors, manufacturing plants, power grids, oil & gas industry, etc. In contrast, IT networks traditionally facilitated business applications based on data received from OT applications. With increased automation, and growing demand for remote operations, it is imminent that the two technology domains need to interwork seamlessly and reliably.¶
Due to lack of coordination between industrial networks and IETF technologies, their evolution priorities have been different and as a result, current IETF technologies and protocols are not well adapted in industrial networks. Industrial systems and applications are becoming increasingly complex and proprietary as emerging use cases require a higher integration of OT-IT functions.¶
The OT networks are often tied to a set of non-internet protocols such as Modbus, Profibus, CANbus, Profinet, etc [SURV]. There are more than 100 different protocols each with it's own packet format and are used in the industry. On the other side inventory management, analytics, monitoring, supply chain and simulation software are part of IT and use IP based technologies.¶
No two industry sectors are same and present different requirements and challenges on the networks. These differences are even more enhanced in industry automation and operations. The processes, control operations, environmental conditions, frequency and type data collection vary across each sector. Yet, there is a need for common, interoperable, off-the-shelf mechanisms and protocols so that applications can be deployed in relatively shorter time.¶
This document provides a framework called 'Integrated Industrial Networks' (or IIN for short) and a discussion on integration of process control, monitoring and operations with IT. It proposes (a) an idea about integrated industrial network stack that would support functions and capabilities from both OT and IT systems, (b) a structured deployment considerations, (c) alignment and coordination across stakeholders from other consortia and SDOs.¶
In this framework a greater focus is on capturing functional and operational requirements for the emerging use cases. The top three considerations are - first, to identify the components required to fulfill the needs for both IT and OT applications. Second, Integrated Industrial Network (IIN) Framework needs to meet and adapt to evolving deployment strategies that include cloud and edge technologies. Finally, mechanisms to coordinate with stakeholders (domain experts) should also be identified when discussing such a framework.¶
Industrial Networks are a combination of technologies that provide capability for the delivery of process control data to/from (and across) the machines and sensors to different controllers and other application specific servers. Thus, in IIN stack, one end often be an OT device and other end an IT function.¶
In OT Systems traditionally,¶
In addition, there is also an emergence of new use cases and scenarios in OT:¶
Some of the reasons why leveraging IETF technologies would be beneficial:¶
A conceptual industry adopted reference model for network segmentation is known by Purdue model or ISA-95 [ISA95]. It shows hierarchical levels through which ordered connectivity between the components (or entities) in Industry Control Systems (ICS) is established. These levels range from 0 at the lowest level for the physical devices to applications at level 5. Those levels also include other control and management equipments (potentially treated as in-network functions and capabilities).¶
| +-------------------------------+ Enterprise | L5 | Enterprise applications | Security +-- +-------------------------------+ Zone | +-------------------------------+ | L4 | Gateways, servers (ops, mgmt) | IDMZ +-- +-------------------------------+ | +-------------------------------+ | L3 | Supervisory controls | Industry | +-------------------------------+ Security | L1 | Device control | Zone | +-------------------------------+ | L0 |Sensors, Actuators, Robots, etc| (cells or zones) +-- +-------------------------------+
The scope and functions in each zone in [ISA95] are summarized below:¶
Consists of Levels 0 through 3 site wide production system.¶
Effectively, industrial networks are under a single administrative control or a limited group of administrators. They are expected to extend across different geographies and over a range of distances.¶
RFC 8799 [LDN] introduces a formal structure and taxonomy to describe large-scale private networks called limited domains. LDNs use public Internet for connectivity across multiple sites and adhere to Internet protocols. However, with in a site, it is acceptable to use proprietary protocols. Thus, an LDN comprises of Internal, External and Boundary protocols.¶
Industrial networks also extend to multiple sites. The enterprise services will reside in the cloud or edges, while factory floors or plants are at remote locations. Structurally, ISA architecture (Figure 1) can be expressed as IETF's Limited Domain Networks [LDN] framework. Thus, L4 and L5 levels in enterprise zone are one site, connected via global Internet to L3-L0 levels at factory floors or plants. Using LDN model, we get break down the framework requirements systematically:¶
The paradigms of networking in OT are quite different than IP based best-effort networking protocols. Yet, IETF protocols are extensively used in OT applications. Often, it is not possible to get contributors directly from the OT sectors, then it would make more sense to coordinate with well-established consortia where OT scenarios and requirements are is discussed may be utilized. Two well established foundations are IIC [IIC] and OPC-UA [OPC]. For example, a [IIC_TALK] provided overview of IIC activities.¶
Industrial IoT Consortium (IIC) provides use cases, scenarios, and best-practice frameworks to solve specific problems and solution pain points. It is a rich resources of case studies and demonstrations of different test beds. The IIC itself is not involved in standards development, but may help in formalizing requirements, further insights into solutions developed in IETF, and potentially help adoption of those solutions.¶
Open Platform Communications-Unified Architecture (OPC-UA) provides interoperability across different hardware platforms using a standard data model. It standardizes various information models, corresponding client-server architecture and defines necessary access mechanisms to those information models. The OPC-UA is an abstraction layer to provide common interface to different data look-up and event notifications. A number of information models are provided by OPC-UA can be found here [OPC_INFO]. Foe example, OPC has a specification on PLCs. It abstracts PLC specific protocols (such as Modbus, Profibus, etc.) into a standardized interface allowing HMI/SCADA systems to interface with a middleware that converts generic-OPC read/write requests into device-specific requests and vice-versa.¶
IETF solutions will focus on leveraging or extending IETF technologies for IT and OT integration which is at the infrastructure or communication layer. Thus, providing protocols that could potentially benefit higher-level OPC-UA work.¶
Both IIC and OPC could provide guidance to the lower level work.¶
Traditionally, OT and IT experts have focussed on different concerns. On a production floor or with OT, the focus is generally on no-congestion, lossless reliable transmission, and real-time or deterministic communication. Quality of manufactured goods, and efficiency of processes is also an important concern for OT experts.¶
With Industry 4.0 initiatives (such as smart factory and smart manufacturing), these concerns are beginning to overlap, i.e. OT networks are also required to be concerned with scalability, security, operations and maintenance from remote locations.¶
The fundamental requirement for industrial networks is to support legacy devices (even when the network infrastructure is upgraded) while enabling emerging applications.¶
Requirements from legacy device support:¶
Requirements from Emerging Trends:¶
Perimeter of device control is expanding from factory floors to the cloud. It is anticipated that Industrial IoT controls when extended to the cloud or edge compute platforms will offer better integration with sophisticated business logic application architectures.¶
With adoption of virtualization several of supervisory or management equipment could transition to IT infrastructure. It may or may not remain on-premises. All scenarios are possible - moving L1,L2, L3 to separate IT network on the same floor, to the edge or to the cloud. Now extending the communication to the edge and cloud nodes increases the distance requiring adoption of layer 3 network designs.¶
Shorter addresses are inherent to industry control systems to provide implicit determinism. For this purpose, the industrial networks use fieldbus interface with the controllers.¶
To develop further on different type of address format support. From smaller address of legacy devices to IT based applications with IP address.¶
(OT-Address)--->(Industry Control)--->(IP-Address) (control dev) ( network ) (application)¶
Preferably allow OT devices to understand IP-addresses for the servers they connect to.¶
Above mentioned emerging trends such as virtualization of PLCs or moving MES or HMI into the cloud will have a significant impact on the framework. It moves functions from manufacturing zone to the cloud which not only influences how latency, safety and resiliency can be assured but also moves the security zones.¶
site-A
+--(L0-L4)----+ (L4-L5) *L1,L2,L3*
| +-----------|-+ +-------------+
| |Manufacturing|------<Detnet>-------|Enterprise |
__| |Zones |-|---------------------|Zone |
| | |(IINS proto) |<BP> <BP> | (IETF proto)|
| +-|-----------+ | | |
| +-|-+- site B-+ (L1) *L3-L4* +-------------+
|_TSN____| |<BP> +--------------+ |
+-------- | Edge Services|---------+
| +--------------+ |
+---------------Limited Domain Network ----------------+
of an industry vertical
In Figure 2, LDN taxonomy of internal, external and boundary protocols is used. The round brackets represent current Purdue model levels. Note that both Manufacturing and Enterprise zones are 'inside protocols' in LDN terminology but can (or may) run different protocol stacks. Each zone may deploy either custom of standard protocols. They interact using outside protocols i.e., public Internet technologies. The translation from inside to outside protocol happens through boundary protocols (shown as <BP> in the figure).¶
Site A Site B Site C
<---------> <----------> <--------->
+-----+ +-----+ +----------+
| L0 |---L0_L1--| L1 |----L1_L2---|L2--L2_X--L5|
+-----+ +-----+ +----------+
Site-Y
<-----------> <---------------------------->
Site M
<------------------------->
Figure 3 above depicts that in IIN framework, boundaries between the interfaces should not be crossed. Moreover, equipment or functions from different levels may be placed at different sites, but in this framework direct communication from higher levels to devices is not permitted.¶
Each interface has at least three attributes associated - whether a particular request is authorized, the service level guarantees (latency, data rate, frequency, etc), security profile.¶
In Figure 3, a level based hierarchical co-location is shown to be preserved.¶
These functions apply to end nodes as well as network nodes or other gateways in the network.¶
The topologies in the manufacturing zones do not change very frequently and devices are also designed for long-term use with minimal time between the failures. Such design considerations may be used to simply network operations and configurations.¶
Assuming this is a layer 3 network architecture, there should be an assignment and association between the network address and end devices' physical addresses. Note that legacy devices are either on serial bus or their information is carried over Ethernet media.¶
Further motivation and analysis for adapting to OT/IT asymmetric address formats is covered in [I-D.draft-km-industrial-internet-requirements].¶
Additionally, adapting these devices to network layer requires support for the following mechanisms:¶
Currently, L0 and L1 devices do not use any transport protocol. The data is embedded after control header. With a network layer solution, TCP maybe too heavy for field-bus devices. Some other means of assuring device delivery will be needed.¶
Routing protocols will be necessary as the scale of the devices grow at the same time it should be kept simple. Possibly, Interior Gateway Protocol (IGP) will be deployed. Here it may be useful to provide guidelines on IGP features that provide distribution of routes (for different devices), path information.¶
Differentiating traffic and assigning priorities is required so that important data is not dropped. This is in addition to use of Detnet for time-sensitive services.¶
Different type of data can include - process data (high priority), monitoring data (low priority), fault, alarms, signals data (high), health-check sensors data (medium), etc.¶
This document requires no actions from IANA.¶
This document introduces no new security issues.¶