Still more on ‘cutting the copper’

One evening, an easy method to scale back on the amount of copper we were installing dawned upon us—a concept that was certainly not revolutionary, but simply logical.


For about 15 years now, I’ve been hanging out with an outstanding group of consulting engineering firms, contractors, and owners who focus solely on the design, construction, and operation of large data centers. 

Somewhere around 1997, when load growth in data centers so suddenly accelerated, we began to realize how much copper we were installing. One evening, over some Scotch, an easy method to scale back dawned upon us—a concept that was certainly not revolutionary, but simply logical. 

We realized that if we just changed our infrastructure electrical design from distributing power at 480 V to distributing instead at 600 V, we could push 25% more power into the facility through the same size conductors and duct banks. 

Conversely, we also realized that we could deliver the same amount of power into the facility through smaller cables, smaller circuit breakers, and lighter switchgear bus—by making just that one simple change. 

The optimal end result turned out to be a combination of both of the above methods, and there was as much cost reduction in switchgear and breakers as there was from reduction in duct bank ampacities. What would have otherwise become boatloads of pricey 5000-amp drawout circuit breakers and 5000 A switchgear bus at 480 V, could remain as 3000 A and 4000 A breakers and bus, but instead at 600 V. The savings in installed cost was tremendous. 

Everyone on the team was a bit apprehensive at first that changing from 480 V to 600 V could stretch cable insulation and switchgear and breakers to their dielectric limits and cause insulation failures—but it all worked great, without any such problems. We then designed and built our next dozen or so large data centers in this fashion, using 600-V switchgear, 600-V distribution transformer secondary windings, 600-V generators, 600-V UPS, and chillers and pump motors and CRAH units. 

The concept was great while it lasted—until loads grew dramatically again, and we found ourselves bumping up against the ceiling once more, and having to install too much copper. 

In one respect, the overall direction in the data center industry seems to be going backward lately. The recent pronounced trend toward operating servers on the data floors at 230 Vac, supplied from 400 V WYE/230 V distribution, is pushing the gains back in the opposite direction. With duct bank heating de-rating factors applied, a large trunk feeder operating at 400 V will require about 60% more tons of copper than a feeder operating at 600 V, to transmit the equivalent block of power from one place to another. 

Moreover, since the 400 Y/230 V distribution arrangement requires at least a full size neutral (and sometimes, an OVERSIZE neutral), the 400-V option typically will require DOUBLE the tonnage of copper, compared to the 600-V three-wire trunk feeders we’ve been using. 

Beyond that, the lower operating voltages of substation transformers and generators is now generally causing significant increases in short-circuit duties throughout these systems, and increasing arc-flash hazard everywhere in the facility. 

A 2500 kVA substation transformer with a 400-V WYE secondary, having a standard IEEE/ANSI impedance in the 5.75% range can supply a heckuva lot of secondary short-circuit current, especially when operating momentarily in parallel with a 2.5M VA 400-V generator. That tends to result in larger fault duties than anyone would want, all the way to the distribution inside the computer rooms—and, all the way down to the rack level. 

So now, in many cases, we’re back not only to 5000-amp breakers and bus, but to 5000 A 4-pole breakers, and a full neutral bus in the switchgear (I got a call from a major switchgear manufacturer a while back, asking “…do you have any idea why you’d want to have an 8000-amp neutral bus in lineups of 400/230-V switchgear, and do you have any idea how we would BUILD that…??”). 

The objective of using 400 V WYE distribution in a data center is clear, in allowing the elimination of one entire level of transformations. And, the particular level of transformation that is eliminated by using the approach happens to also eliminate one of the greatest efficiency “bottlenecks” in a large data center. But, there are obviously trades-offs to be made in changing to distribution at 400 V. 

In my opinion, it’s time to re-visit the loadenter unit substation concept we’ve been talking about for so many weeks, and consider now taking the 400, 480, and 600 V distribution voltages up to much higher voltages.

There are a lot of smart folks looking at this at this very moment. A number of new developments are coming together at the same time, including recent innovations in medium-voltage switchgear design and medium-voltage UPS systems. 

These lead me to believe that infrastructure distribution at a nominal voltage like 4160 V will become the “New 480 V” in power distribution in large data centers, in the not-too-distant future. Even nominal system voltages that haven’t been very common in a while—say, for example, 2400 V—could return to popularity. 

Here’s an interesting presentation, entitled “Medium Voltage Electrical Systems For Data Centers”  from Michael Mosman, PE, CTO of CCG Facilities Integration, that nicely frames the discussion. 

I expect that the liquid-filled HDC substation transformers we’ve been discussing for so many weeks will also figure prominently in this “next-generation” of large data centers, soon to come. More about this next week.

Anonymous , 03/14/13 11:34 AM:

Sounds like the building has a very dry basement with
machinery rooms,and roof fan rooms? Safety is more important then a 100 lbs of
copper bus bars.
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2013 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
A cool solution: Collaboration, chemistry leads to foundry coat product development; See the 2015 Product of the Year Finalists
Raising the standard: What's new with NFPA 70E; A global view of manufacturing; Maintenance data; Fit bearings properly
Sister act: Building on their father's legacy, a new generation moves Bales Metal Surface Solutions forward; Meet the 2015 Engineering Leaders Under 40
Cyber security cost-efficient for industrial control systems; Extracting full value from operational data; Managing cyber security risks
Drilling for Big Data: Managing the flow of information; Big data drilldown series: Challenge and opportunity; OT to IT: Creating a circle of improvement; Industry loses best workers, again
Pipeline vulnerabilities? Securing hydrocarbon transit; Predictive analytics hit the mainstream; Dirty pipelines decrease flow, production—pig your line; Ensuring pipeline physical and cyber security
Upgrading secondary control systems; Keeping enclosures conditioned; Diagnostics increase equipment uptime; Mechatronics simplifies machine design
Designing positive-energy buildings; Ensuring power quality; Complying with NFPA 110; Minimizing arc flash hazards
Building high availability into industrial computers; Of key metrics and myth busting; The truth about five common VFD myths

Annual Salary Survey

After almost a decade of uncertainty, the confidence of plant floor managers is soaring. Even with a number of challenges and while implementing new technologies, there is a renewed sense of optimism among plant managers about their business and their future.

The respondents to the 2014 Plant Engineering Salary Survey come from throughout the U.S. and serve a variety of industries, but they are uniform in their optimism about manufacturing. This year’s survey found 79% consider manufacturing a secure career. That’s up from 75% in 2013 and significantly higher than the 63% figure when Plant Engineering first started asking that question a decade ago.

Read more: 2014 Salary Survey: Confidence rises amid the challenges

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.