Mitigate control panel security, safety risks
A review of relevant standards, including UL 1436 and NFPA 70E.
Integrating security into equipment and enclosures is essential. It allows design/controls engineers to create multiple layers of security that work together to protect the network and equipment without retrofitting or replacing components. Physical security solutions save time and costs associated with security breaches, network downtime, repairs and hardware replacement due to theft.
Significant transformations are taking place in manufacturing. Having access to data and analytics throughout the product lifecycle, from inception to order management and fulfillment is key to Industry 4.0. At the base of the data access design model is the physical network or communication infrastructure. Vulnerabilities are easily identified at the physical layers and extend to all aspects of the system.
With integrated factory systems access to almost all data is available in near real time. This is a blessing and a curse. A blessing that we have the possibilities of seeing lot information and origination records when we need them. A curse because accessibility implies open standards, availability of data at all levels of the enterprise with vulnerabilities, especially when the networks are not protected with security layers.
Securing the data infrastructure is not a goal that can be reached one time at the design stage and then forgotten. It is an ongoing process that must start with a solid network design that continues to live on throughout the manufacturing lifecycle.
Older systems are at a higher security risk as older control and automation designs did not account for, or plan on, security or vulnerabilities that continue to evolve with new applications. These older systems represent added risks for control engineers who are looking to adapt new technologies in brownfield, existing systems.
Production data flow and security
Production data originates at the edge device. Computing is done at the edge application layer, for real-time response to changes in nearby devices and sensors. Post-processing and housing of the data is supported by the control application layer, which includes higher-level functions, local reporting and recipe management. Some data is stored and acted on locally by the servers, but most all higher-level functions are performed in the control layer of the architecture. Therefore, securing the edge layer from attacks is key to the secure functions of the overall system (see Figure 1).
Data flow from the factory network to the enterprise layer is selective and purposeful. Data sent to the enterprise layers (cloud or otherwise enterprise data centers) is destined for additional post processing, historical analytics, higher-level functions reporting, big data management and storage.
It is clear, that if data does not originate cleanly from the edge in a timely fashion, none of the higher-level analytics and big data management functions can occur in a timely and reactive way to improve corporate metrics such as overall equipment effectiveness (OEE), materials resource planning (MRP), customer relationship management (CRM) and so on. At all levels of the infrastructure; security concerns, product life cycle issues, data safety and a long list of vulnerabilities are analyzed and measured throughout the enterprise to maintain a delicate balance between productivity, profitability, customer satisfaction and safety.
Security is a main concern at all levels of the physical network, or infrastructure. All inputs are considered security threats in the data aggregation model. The system is as strong as its weakest point. Therefore, it is important to look at the vulnerability of a system during the design process and remove known weak spots by providing designed-in layers of protection. Security is considered the first line of defense that must be recognized, designed, built and later monitored to assure continuous security compliance. Absolute security is not a realistic goal as threats evolve continuously over time. Physical security solutions should start at the edge of the network and be found throughout the physical network layers.
Physical security at the edge:
- Data ports are protected with physical security measures that include physical hardware keys to prevent tampering with network traffic (see Figure 2).
- Select networks, or all network ports, are locked at the insertion point to prevent network disruption by protecting the RJ45 connectors with patch cord lock-in devices.
- Colored data ports are keyed to prevent mixing network data flows (see Figure 3). Network segmentation of colored/keyed data-ports cannot be modified after the initial network design and deployment. Colored/keyed ports with fixed IP addressing give the control engineers complete control over the network segments and data flow over the life of the control system.
- Network aggregation points are secured with physical coded access entry keys that keep unauthorized personnel out of the network enclosures and data systems (see Figure 4).
Some of these physical security measures can also be incorporated in existing brownfield automation and control systems to limit vulnerabilities of older control systems.
Media at the edge:
Choosing the right network media (copper, fiber, network enclosures and pathways) play another level of functional security and data safety also considered early in the system design process.
- Fiber network segments are immune to electromagnetic interference (EMI) and electrical noise and can be used for long network segments where systems are scattered over large fields (i.e., oil and gas refinery, water or wastewater sites, mining operations). Fiber optic network lines also can be tampered proof or tampered evident. This means that fiber traffic cannot be intercepted or tampered with without affecting the traffic, hence affecting immediately the state and health of the data network. Fiber optic network lines are generally used in horizontal (permanent) network deployment runs for all these reasons.
- Copper communication cables are easier to deploy to edge devices, since most equipment providers have copper connectivity at the device level. The choice of the right copper cables and shielding can play an important role in securing the data at the edge. TIA-1005-A M.I.C.E specifications are especially important considerations at this level of the network. Mechanical, ingress, chemical and electromagnetics are all hazards that can easily affect the data flow and security at the edge. Protecting network systems from these hazards means complete system replacement of the network. The choice of network components including cabling, shielding and pathways is critical during the inception stages of good communication networks.
- Cable management is a key consideration of the physical infrastructure for securing the cable runs, optimizing system reliability, effective space utilization and scalability (see Figure 5). It is important to consider industry-leading cable routing systems as part of comprehensive, integrated data structured solutions to effectively manage and protect high-performance communication, computing and power systems. Well-planned overhead cable routing system, which can include multiple layers of protections against MICE environmental concerns, contribute to effective real estate usage, additional network physical security layer and network performance. Cable pathways provide network structural integrity and cable protection. The choices of cable conduit and pathways for the exclusive use of telecommunications equipment provide proper environment and adequate security to the data network.
Safety at the edge:
- Network enclosures are major components of the physical network security system. Enclosures protect against environmental and human hazards. These devices protect the network in many ways by keeping the system integrity, especially when network traffic is isolated from power systems where EMI noise levels are at their highest (see Figure 6). Some automation system designers integrate communication networks inside power enclosures like drive systems or power distribution closets. This approach is not advisable for reliable long-term system operations because of the EMI noise, which will ultimately affect network traffic, data integrity and system operations. The best network enclosure is independent from the automation and control system with clean UPS power, fiber and copper patching systems to allow connectivity to edge and system level servers in the control room and away from the edge devices.
- If/when communication networks are included in the industrial control enclosures where there are higher EMI noise levels and voltages that present a shock or arc flash hazards are present, qualified, authorized electrical workers should follow NFPA 70E: Standard for Electrical Safety in the Workplace when accessing the electrical enclosure and control systems. Within NFPA 70E Article 120.5 (7), the process of verifying the absence of voltage is defined as a qualified worker with a portable test instrument (voltmeter), using the required personal protective equipment (PPE), and testing for voltage on each phase conductor or circuit part to mitigate shock or arc-flash hazards. This traditional method of voltage verification has the potential for human error and other limitations. Consequently, in the 2018 edition, NFPA 70E Article 120.5 (7) Exception 1 was added to provide an alternate method of verifying the absence of voltage through a new product category: Absence of voltage testers (AVTs). A line of AVT devices that are permanently installed in control panel and power distribution enclosures to test for the absence of voltage is now available (see Figure 7). These AVT devices test for both ac and dc voltage and therefore will see when capacitive voltage is present. These UL1436/SIL3-rated AVT testers mitigate electrical risks associated with the task of verifying the absence of voltage by reliably automating the testing process without exposing workers to the electrical hazards.
- Alternatively, to access the communication ports (USB or RJ-45) of the control system while under full power, and to mitigate the shock and arc flash electrical hazards, it is advisable to connect to the communication network through regulatory approved connection points while the enclosure is energized and closed in normal operating conditions. For this reason, it is preferable to add data access ports to the enclosure doors providing operation personnel access to the communication and power for monitoring, troubleshooting and other maintenance tasks while lowering the risks associated with the electrical hazard. Data-access ports are installed and configurable with the right combination of data and power ports to solve this dilemma. Figure 8 shows a data-access port with USB, RJ45 and GFCI power outlets with IP65/66 ingress protection that may be locked while the system is not in use by operators or maintenance personnel.
Monitoring the edge
Physical security always must be maintained. To do so, telecommunications networks must be monitored and audited with regularity. Well-planned network design is a good start, but security concerns are always present and real. Human operators and M.I.C.E-identified environmental hazards play major role in the health of the network over time. Monitoring network software platforms are deployed to oversee the network operations in real time and can provide health reporting data 24/7. In ideal system operations, nothing should affect system operations, but factories run in suboptimal conditions; additional network components, foreign components and many external devices, find their way to open ports or unmonitored Wi-Fi channels on the network. Added devices are security threats need to be identified, monitored and prevented from accessing traffic bandwidth of normal system operations. Monitoring software applications can immediately identify these security threats and limit or remove them by blocking the affected ports and alerting system operators of the imminent threat to the physical communication network (see Figure 9).
The data network should be considered as an integral part of the overall security strategy; it starts at the network components of the physical layer but never ends there. Integrating security into the communication network is essential as it allows design/controls engineers to create multiple layers of security that extend outside the immediate automation and control scope of controlling machines and assets to protect the network and equipment. Physical security solutions allow companies to save time and costs associated with security breaches, network downtime, repairs and hardware replacement. Considerations must be made for system operations and data protection throughout the lifecycle of the manufacturing system.
Assessments must be periodically planned into the process to revisit the integrity of the network operations and systems, with well-planned remediation strategies as required. You should trust the safety of your data network infrastructure to certified network infrastructure designers, engineers and installers. Finally, be safe out there; it is a dangerous world for all manufacturing and product data. Adopt extreme security measures so you are never faced with extreme network outages that can cripple your operations.