Still more on ‘cutting the copper’

One evening, an easy method to scale back on the amount of copper we were installing dawned upon us—a concept that was certainly not revolutionary, but simply logical.

09/19/2012


For about 15 years now, I’ve been hanging out with an outstanding group of consulting engineering firms, contractors, and owners who focus solely on the design, construction, and operation of large data centers. 

Somewhere around 1997, when load growth in data centers so suddenly accelerated, we began to realize how much copper we were installing. One evening, over some Scotch, an easy method to scale back dawned upon us—a concept that was certainly not revolutionary, but simply logical. 

We realized that if we just changed our infrastructure electrical design from distributing power at 480 V to distributing instead at 600 V, we could push 25% more power into the facility through the same size conductors and duct banks. 

Conversely, we also realized that we could deliver the same amount of power into the facility through smaller cables, smaller circuit breakers, and lighter switchgear bus—by making just that one simple change. 

The optimal end result turned out to be a combination of both of the above methods, and there was as much cost reduction in switchgear and breakers as there was from reduction in duct bank ampacities. What would have otherwise become boatloads of pricey 5000-amp drawout circuit breakers and 5000 A switchgear bus at 480 V, could remain as 3000 A and 4000 A breakers and bus, but instead at 600 V. The savings in installed cost was tremendous. 

Everyone on the team was a bit apprehensive at first that changing from 480 V to 600 V could stretch cable insulation and switchgear and breakers to their dielectric limits and cause insulation failures—but it all worked great, without any such problems. We then designed and built our next dozen or so large data centers in this fashion, using 600-V switchgear, 600-V distribution transformer secondary windings, 600-V generators, 600-V UPS, and chillers and pump motors and CRAH units. 

The concept was great while it lasted—until loads grew dramatically again, and we found ourselves bumping up against the ceiling once more, and having to install too much copper. 

In one respect, the overall direction in the data center industry seems to be going backward lately. The recent pronounced trend toward operating servers on the data floors at 230 Vac, supplied from 400 V WYE/230 V distribution, is pushing the gains back in the opposite direction. With duct bank heating de-rating factors applied, a large trunk feeder operating at 400 V will require about 60% more tons of copper than a feeder operating at 600 V, to transmit the equivalent block of power from one place to another. 

Moreover, since the 400 Y/230 V distribution arrangement requires at least a full size neutral (and sometimes, an OVERSIZE neutral), the 400-V option typically will require DOUBLE the tonnage of copper, compared to the 600-V three-wire trunk feeders we’ve been using. 

Beyond that, the lower operating voltages of substation transformers and generators is now generally causing significant increases in short-circuit duties throughout these systems, and increasing arc-flash hazard everywhere in the facility. 

A 2500 kVA substation transformer with a 400-V WYE secondary, having a standard IEEE/ANSI impedance in the 5.75% range can supply a heckuva lot of secondary short-circuit current, especially when operating momentarily in parallel with a 2.5M VA 400-V generator. That tends to result in larger fault duties than anyone would want, all the way to the distribution inside the computer rooms—and, all the way down to the rack level. 

So now, in many cases, we’re back not only to 5000-amp breakers and bus, but to 5000 A 4-pole breakers, and a full neutral bus in the switchgear (I got a call from a major switchgear manufacturer a while back, asking “…do you have any idea why you’d want to have an 8000-amp neutral bus in lineups of 400/230-V switchgear, and do you have any idea how we would BUILD that…??”). 

The objective of using 400 V WYE distribution in a data center is clear, in allowing the elimination of one entire level of transformations. And, the particular level of transformation that is eliminated by using the approach happens to also eliminate one of the greatest efficiency “bottlenecks” in a large data center. But, there are obviously trades-offs to be made in changing to distribution at 400 V. 

In my opinion, it’s time to re-visit the loadenter unit substation concept we’ve been talking about for so many weeks, and consider now taking the 400, 480, and 600 V distribution voltages up to much higher voltages.

There are a lot of smart folks looking at this at this very moment. A number of new developments are coming together at the same time, including recent innovations in medium-voltage switchgear design and medium-voltage UPS systems. 

These lead me to believe that infrastructure distribution at a nominal voltage like 4160 V will become the “New 480 V” in power distribution in large data centers, in the not-too-distant future. Even nominal system voltages that haven’t been very common in a while—say, for example, 2400 V—could return to popularity. 

Here’s an interesting presentation, entitled “Medium Voltage Electrical Systems For Data Centers”  from Michael Mosman, PE, CTO of CCG Facilities Integration, that nicely frames the discussion. 

I expect that the liquid-filled HDC substation transformers we’ve been discussing for so many weeks will also figure prominently in this “next-generation” of large data centers, soon to come. More about this next week.



Anonymous , 03/14/13 11:34 AM:

Sounds like the building has a very dry basement with
machinery rooms,and roof fan rooms? Safety is more important then a 100 lbs of
copper bus bars.
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2013 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
The true cost of lubrication: Three keys to consider when evaluating oils; Plant Engineering Lubrication Guide; 11 ways to protect bearing assets; Is lubrication part of your KPIs?
Contract maintenance: 5 ways to keep things humming while keeping an eye on costs; Pneumatic systems; Energy monitoring; The sixth 'S' is safety
Transport your data: Supply chain information critical to operational excellence; High-voltage faults; Portable cooling; Safety automation isn't automatic
Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Plant Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.

Maintaining low data center PUE; Using eco mode in UPS systems; Commissioning electrical and power systems; Exploring dc power distribution alternatives
Synchronizing industrial Ethernet networks; Selecting protocol conversion gateways; Integrating HMIs with PLCs and PACs
Why manufacturers need to see energy in a different light: Current approaches to energy management yield quick savings, but leave plant managers searching for ways of improving on those early gains.

Annual Salary Survey

Participate in the 2013 Salary Survey

In a year when manufacturing continued to lead the economic rebound, it makes sense that plant manager bonuses rebounded. Plant Engineering’s annual Salary Survey shows both wages and bonuses rose in 2012 after a retreat the year before.

Average salary across all job titles for plant floor management rose 3.5% to $95,446, and bonus compensation jumped to $15,162, a 4.2% increase from the 2010 level and double the 2011 total, which showed a sharp drop in bonus.

2012 Salary Survey Analysis

2012 Salary Survey Results

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.