Meeting electrical infrastructure demands in data centers

A data center’s electrical system requires a robust, reliable infrastructure that far surpasses that of its commercial and industrial facility peers.

05/16/2013


Learning objectives

  • Understand today’s data center demands and how to meet them.
  • Know the requirements for key equipment and its installation.
  • Understand how to properly specify wiring for various voltages.

To describe a data center using an analogy, a data center is a womb with no view—for computers. Designed to make complex equipment comfortable, the data center requires a robust and highly reliable electrical infrastructure that far surpasses that of its commercial and industrial facility peers.

These high-reliability infrastructure differences are achieved through meeting unique operational efficiencies, properly selecting and installing electrical equipment, and specifying proper wiring and design methods with appropriate voltages, all while meeting maintainability requirements for planned maintenance.

The first step in this process is to define key electrical system base requirements/data center goals. These are typical for a high-reliability installation:

Bus ducts in data centers have many joints, typically found every 10 ft in straight runs, resulting in equipment reliability and concurrent maintainability issues. Therefore, precise design is critical. Courtesy: Universal Electric Corp.1. Redundant components and systems are equivalent to a person leaving the house in the morning with pants that are too big, so he grabs a belt and a pair of suspenders. If the belt breaks, the suspenders will keep the pants up and vice versa. Either way, he’s covered.

2. Concurrent maintainability means assuring that every component and system (both power and cooling) that supplies the computers can be taken out of service for replacement, repair, or maintenance without shutting the computers down.

3. Fault tolerance, different from concurrent maintainability, means that when any component or system breaks or fails, the systems automatically reconfigure so that the computers don’t shut down. Fault tolerance is an automatic process; concurrent maintainability is a manual process. Part of fault tolerance is compartmentation so that a fire or explosion in one area does not result in total loss of power, cooling, or both to the computers.

4. Complete standby power backup is achieved with a generator plant that is set to provide power when the utility company is not available.

5. Selective overcurrent coordination of circuit breakers and/or fuses is achieved so that during a fault, only the minimum amount of the system is shut down. Ideally, the system will open only the circuit breakers that supply the single piece of failed equipment and nothing else upstream.

6. Modular, scalable construction allows a data center to expand down the road without overbuilding capacity on day one. This is crucial for two reasons: First, everyone is watching their pocketbooks, so if 10 MW of computers will eventually be needed but only 5 MW is needed on day one, the total cost of ownership (TCO) can be minimized by building a modular, scalable shell for 10 MW, but only 5 MW of interior infrastructure for day one. Second, the modular, scalable data center is easier to maintain. Data centers with an excess of unused capacity are a maintenance headache. Careful consideration of the ultimate facility configuration and expansion phases is necessary to minimize risk and eliminate the need for computer equipment shutdown during expansion.

7. Underground circuits are employed in data centers for two reasons: Contractors think they are less expensive to install, and they provide physical security and compartmentation for the data center’s wiring system. However, it is important to note that they require special calculations during the design phase. The Neher-McGrath calculations, found in the National Electrical Code (NEC) 310.15.C and ANNEX B, must be used to design all underground circuits. These calculations often result in the number and size of conductors run underground being substantially larger than would be required overhead. Thus, the anticipated economy compared to overhead circuits is often a false hope.

8. Emphasis on operating efficiency (reduced operating expense, or OPEX) and minimizing TCO can be met by reducing power usage effectiveness (PUE).

Each one of these requirements/goals is crucial because, unlike the typical commercial or industrial facility, the data center load is continuous, with increased ambient temperatures in many areas. For example, the back sections of data cabinets can be 104 to 113 F where branch circuit wiring is installed, while hot aisles can reach the same 104 to 113 F where branch circuit wiring is run upstream from the cabinets. These elevated temperatures result from higher supply air temperature to the computer equipment as a strategy to reduce PUE. Electrical equipment rooms (except those containing storage batteries) can operate at up to 104 F to reduce PUE. The extreme temperatures of a data center make its design for high operating temperatures in addition to code requirements that much more critical than a typical commercial or industrial facility design.

Operations and maintenance

An alternative to bus duct, the cable is assembled like a cable tray with large single-conductor power cables run within it and can be easily modified in the field to fit field conditions. With only two terminations (one at each end, with solid cable in bBeyond unique base design requirements, skilled data center designers also must consider equipment maintenance during design, as ease of maintenance will be crucial to meeting continuous and reliable data center operations. Because a lot of maintenance is required to sustain the critical environment, concurrent maintainability, arc flash labeling, and mean time to repair (MTTR) reduction all play a role in maintaining a data center’s electrical operations.

Designing the data center’s electrical systems to achieve concurrent maintainability means creating an arrangement where any piece of equipment or system that supplies the computers can be taken offline for maintenance purposes while the load continues to operate.

On occasion, maintenance is performed on a piece of equipment while it’s energized (hot work). While a high-reliability data center is designed for concurrent maintainability, some operators choose hot work to reduce maintenance time. While there are a lot of safety procedures in place for this type of maintenance, the best way to understand the risks associated with each piece of data center equipment is to understand its arc flash labeling. This label reflects the arc flash hazard calculated for each piece of equipment and identifies the level of personal protective equipment (PPE) and distances required for safe maintenance. It’s important to understand that some maintenance on small parts in restricted places cannot be performed with NFPA 70E levels 3 and 4 PPE.

Minimizing the time it takes to repair a piece of crucial data center electrical equipment and get it back in service to meet the load’s needs (MTTR) also is important in calculating data center maintenance in advance and when specifying equipment. Proper specification can reduce MTTR. For example, a 4000 A, low voltage, drawout-mounted circuit breaker can be withdrawn and replaced from shelf stock in 15 minutes, while a similar stationary-mounted circuit breaker might take an hour or more to replace.


<< First < Previous 1 2 3 Next > Last >>

HEINZ , CA, United States, 06/06/13 06:35 PM:

In the example on P40:
For a continuous load of 400A, the NEC requires a feeder that can carry 1.25 the continuous 400A = 500A.
So how does a 100% rated CB save on cabling costs.
Also, for a standard 80% rated CB, the terminal temp rating up to 100A is 60*, and 75* for over 100A. What is the temp rating for the terminals of a 100% rated CB?
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2015 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
Your leaks start here: Take a disciplined approach with your hydraulic system; U.S. presence at Hannover Messe a rousing success
Hannover Messe 2016: Taking hold of the future - Partner Country status spotlights U.S. manufacturing; Honoring manufacturing excellence: The 2015 Product of the Year Winners
Inside IIoT: How technology, strategy can improve your operation; Dry media or web scrubber?; Six steps to design a PM program
Getting to the bottom of subsea repairs: Older pipelines need more attention, and operators need a repair strategy; OTC preview; Offshore production difficult - and crucial
Digital oilfields: Integrated HMI/SCADA systems enable smarter data acquisition; Real-world impact of simulation; Electric actuator technology prospers in production fields
Special report: U.S. natural gas; LNG transport technologies evolve to meet market demand; Understanding new methane regulations; Predictive maintenance for gas pipeline compressors
Warehouse winter comfort: The HTHV solution; Cooling with natural gas; Plastics industry booming
Managing automation upgrades, retrofits; Making technical, business sense; Ensuring network cyber security
Designing generator systems; Using online commissioning tools; Selective coordination best practices

Annual Salary Survey

Before the calendar turned, 2016 already had the makings of a pivotal year for manufacturing, and for the world.

There were the big events for the year, including the United States as Partner Country at Hannover Messe in April and the 2016 International Manufacturing Technology Show in Chicago in September. There's also the matter of the U.S. presidential elections in November, which promise to shape policy in manufacturing for years to come.

But the year started with global economic turmoil, as a slowdown in Chinese manufacturing triggered a worldwide stock hiccup that sent values plummeting. The continued plunge in world oil prices has resulted in a slowdown in exploration and, by extension, the manufacture of exploration equipment.

Read more: 2015 Salary Survey

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
This article collection contains several articles on the vital role that compressed air plays in manufacturing plants.
This article collection contains several articles on the Industrial Internet of Things (IIoT) and how it is transforming manufacturing.
This article collection contains several articles on strategic maintenance and understanding all the parts of your plant.
click me