Ethernet network design issues in the substation

Intelligent electronic devices (IEDs) are including Ethernet for communications between each other.


A number of developments translate to the fact that the consulting specifying engineer will need to address myriad issues relating to the use of Ethernet in a substation

More and more products are including Ethernet, which, as you probably know, is an IEEE standard (IEEE 802.3) suite comprising the “seven layer model” in communications created by the ISO, or International Organization for Standardization.

Intelligent electronic devices (IEDs) are including Ethernet for communications between each other. If the CSE is working on an IEC 61850 implementation, that’s built on Ethernet. It would be very common for monitoring and control of that substation to be tied into the substation owner’s Wide Area Network (WAN) or even their Local Area Network (LAN) for, let's say, a large industrial facility. Gone are the days – at least they’re rapidly disappearing – when we ran dedicated copper lines and serial connections. In short, the power industry is moving towards Ethernet connectivity in the substation.

Designing Ethernet for a substation must include functional and environmental factors to perform the tasks that it's being called on to serve. Design considerations for Ethernet include, for instance, the speed of various substation operations and the prioritization of messages. These are critical factors, particularly if that Ethernet network is carrying protective relaying functions.

One of the first things to examine is the survivability of Ethernet products such as switches and routers. You cannot dash to your local big box retailer for a switches and routers used in the home. Ethernet products must have the same environmental survivability characteristics of the other products in that substation. The IEEE standard which lays out the environmental requirements for communications devices in a substation is IEEE 1613.

IEEE 1613 lays out all of the things such as temperature range, susceptibility to surge, voltage supply, cooling, transients, everything that the device might be expected to experience in normal substation operations and continue to operate in the system.

Another important aspect of the network is how the network is configured. Switches and routers have configuration parameters for how they carry traffic, how they route, how they learn – the heuristics of the traffic. These are typically referred to as managed switches and managed routers and they will perform differently depending on how they are configured. Space doesn’t permit digression into network configuration for proper substation operation here, but the issue requires the CSE without background in this subject area to secure the services of a network engineer to figure out how that network should be configured.

Management of the networks is another issue. If the network in the substation is connecting to the WAN or LAN of the facility or any other IT- related facilities, the client’s IT department is going to become involved. That IT department will want an SNMP-enabled product going in and may influence the type of switch or router that's being installed, due to its capabilities, its management tools, etc. SNMP, or Simple Network Management Protocol, is used by and embedded in telecommunications products to enable remote monitoring of, say, the trouble codes of communication devices.

In a utility, this work often brings IT and OT (operations technology) into conflict, as they debate the characteristics, virtues and desired features of IT products going into a substation environment. For the CSE, it is extremely important to understand and to convey to all parties involved that Ethernet products designed into a substation, particularly as part of the protection and control system, must first meet the requirements and the functionality of the protection and control system. Secondarily, it should meet the requirements of the IT department, after the primary requirement is met. Everybody must be on board with the priorities of the device and the selection of the equipment is made accordingly. A simple search on the Web will reveal an array of products designed for substation applications.

The CSE fortunately has an array of choices when determining the media of the Ethernet in the substation; some applications may call for copper, some for fiber optic cable. Fiber has less susceptibility to transience and surges, but unless the Ethernet has a very long run to the apparatus it serves, fiber is viewed as unnecessary given today's class of products. Finally, there’s a question of outside access. There's great interest in connecting to a WAN that’s connected to the Internet for remote access, monitoring and control. While an Ethernet network connected to the Internet provides a tremendous range of possibilities in terms of remote access and control, it also raises cyber security concerns. Thus network design issues will include firewalls and cyber security standards related to those in IEEE 1686.

Sam Sciacca is an active senior member in the IEEE and the International Electrotechnical Commission (IEC) in the area of utility automation. He has more than 25 years of experience in the domestic and international electrical utility industries. Sciacca serves as the chair of two IEEE working groups that focus on cyber security for electric utilities: the Substations Working Group C1 (P1686) and the Power System Relay Committee Working Group H13 (PC37.240). Sciacca also is president of SCS Consulting.

No comments
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2013 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
A cool solution: Collaboration, chemistry leads to foundry coat product development; See the 2015 Product of the Year Finalists
Raising the standard: What's new with NFPA 70E; A global view of manufacturing; Maintenance data; Fit bearings properly
Sister act: Building on their father's legacy, a new generation moves Bales Metal Surface Solutions forward; Meet the 2015 Engineering Leaders Under 40
Cyber security cost-efficient for industrial control systems; Extracting full value from operational data; Managing cyber security risks
Drilling for Big Data: Managing the flow of information; Big data drilldown series: Challenge and opportunity; OT to IT: Creating a circle of improvement; Industry loses best workers, again
Pipeline vulnerabilities? Securing hydrocarbon transit; Predictive analytics hit the mainstream; Dirty pipelines decrease flow, production—pig your line; Ensuring pipeline physical and cyber security
Upgrading secondary control systems; Keeping enclosures conditioned; Diagnostics increase equipment uptime; Mechatronics simplifies machine design
Designing positive-energy buildings; Ensuring power quality; Complying with NFPA 110; Minimizing arc flash hazards
Building high availability into industrial computers; Of key metrics and myth busting; The truth about five common VFD myths

Annual Salary Survey

After almost a decade of uncertainty, the confidence of plant floor managers is soaring. Even with a number of challenges and while implementing new technologies, there is a renewed sense of optimism among plant managers about their business and their future.

The respondents to the 2014 Plant Engineering Salary Survey come from throughout the U.S. and serve a variety of industries, but they are uniform in their optimism about manufacturing. This year’s survey found 79% consider manufacturing a secure career. That’s up from 75% in 2013 and significantly higher than the 63% figure when Plant Engineering first started asking that question a decade ago.

Read more: 2014 Salary Survey: Confidence rises amid the challenges

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.