Averting a data center legal crisis


Design, build, and repair costs

Colin Wicker is a senior attorney in Dorsey & Whitney's construction and design practice group. Courtesy: Dorsey & WhitneyObviously, the engineers who design data centers and the contractors who build and maintain them are aware of the critical importance of continued data center operation. They strive to design and build data centers that physically protect the equipment while reliably and continuously supplying electricity and cooling. They aim to provide robust systems. Depending on the client’s budgets and needs, a data center will also have varying levels of redundancy. Redundant systems and components allow owners to conduct maintenance without having to shut down a data center, and can also keep a data center operating despite a failure in a particular piece of equipment or system. The robustness, reliability, and redundancy of a data center are often discussed in terms of the four-tier system created by the Uptime Institute, an industry consulting firm. A Tier 1 data center is just a server room with nonredundant systems, while a Tier 4 data center has fully redundant subsystems so that it is fault-tolerant and concurrently maintainable.

Unlike in most other commercial projects, the electrical and cooling systems are usually the key cost drivers for a data center project. Taken together, the electrical and HVAC systems can be as much as 75% of the total cost of building a data center. These costs are driven by both the amount of electrical load and cooling required to keep the computers operational and at the correct temperatures, and the redundancies designed and built into those systems to avoid unplanned outages and allow for system maintenance that does not require a shutdown of the IT equipment.

Top tier data centers are expensive. Obviously, the cost of any particular data center will depend on a number of factors including size, locale, efficiency of design and construction, and the level of redundancy in the key building systems. However, industry sources and reports in the media provide some rough idea of the magnitude of the costs incurred by those building data centers. In a 2013 report on a survey of 1,000 data center facilities operators, the Uptime Institute stated that it believed typical data center construction costs are approximately $10 million per MW. In 2011, James Hamilton, a vice president and distinguished engineer at Amazon Web Services, similarly estimated at an Amazon Technology Open House that an 8 MW data center would cost $88 million, which does not differ too much from the Uptime Institute’s rule of thumb. Recent news reports show that the very largest data centers can cost in the hundreds of millions of dollars. Facebook, for instance, is spending $300 million to build the first phase of a three-phase data center in Altoona, Iowa, outside of Des Moines. In that same metro area, Microsoft is spending $700 million to expand one of its data centers. The National Security Agency’s troubled new data center in Bluffdale, Utah, which may be the largest data center in the world, is said to have cost approximately $1.5 billion.

Data centers are not just expensive to design and build. They can also be expensive and difficult to repair. If a data center is hosting critical corporate applications, the corporate owner will want a thorough and careful repair to avoid the costs and disruptions of an additional outage. Sears says in its lawsuit that its Troy, Mich., data center failed after four UPS modules went offline. Sears claims to have been billed more than $2.2 million for subsequent repairs. In addition to those repair costs, Sears also spent more than $500,000 on generator fuel and rental equipment used to keep the data center operational while portions of the electrical system were repaired. The new NSA data center in Utah, which is not fully operational, has already experienced serious problems and is in need of repair. It has experienced 10 arc flash incidents, each of which caused up to $100,000 in equipment damage. Over 50,000 man hours have been spent trying to determine the reason for the repeated and serious problems with the electrical system. The costs to remedy the problems are not yet known as the government and its contractors are still determining the source of the problems and how they can best be fixed, but the costs likely will be significant.

If a data center must be kept operational while it is repaired, that can add significantly to the costs. The safety precautions, planning, and methodical approach necessary to safely carry out repair work while continuing to provide power and cooling services to the computer equipment hosted in the same facility slows down the repair process and adds to the cost. Depending on the nature of the failure, the data center owner may also need to rent temporary equipment or purchase fuel to keep generators running, as Sears did after the failures at its data center in Michigan.

If a data center outage is the result of some party’s negligence or breach of its contractual duties, the potential liabilities can be significant. A party responsible for an outage could face claims for lost profits, repair costs, costs spent putting in place temporary solutions while repairs are conducted, and other miscellaneous costs like fuel and the costs of engineers and IT industry consultants involved in investigating a failure. The potential liability from a data center outage can easily rise into the millions. Sears, for example, is claiming that it has at least $4.9 million in damages as the result of outages at its data center in Michigan. And in 2011, Northrup Grumman spent $4.7 million to settle a lawsuit arising from failures at the State of Virginia’s data center.

The Top Plant program honors outstanding manufacturing facilities in North America. View the 2015 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
Doubling down on digital manufacturing; Data driving predictive maintenance; Electric motors and generators; Rewarding operational improvement
2017 Lubrication Guide; Software tools; Microgrids and energy strategies; Use robots effectively
Prescriptive maintenance; Hannover Messe 2017 recap; Reduce welding errors
The cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Mobility as the means to offshore innovation; Preventing another Deepwater Horizon; ROVs as subsea robots; SCADA and the radio spectrum
Research team developing Tesla coil designs; Implementing wireless process sensing
Commissioning electrical systems; Designing emergency and standby generator systems; Paralleling switchgear generator systems
Natural gas engines; New applications for fuel cells; Large engines become more efficient; Extending boiler life

Annual Salary Survey

Before the calendar turned, 2016 already had the makings of a pivotal year for manufacturing, and for the world.

There were the big events for the year, including the United States as Partner Country at Hannover Messe in April and the 2016 International Manufacturing Technology Show in Chicago in September. There's also the matter of the U.S. presidential elections in November, which promise to shape policy in manufacturing for years to come.

But the year started with global economic turmoil, as a slowdown in Chinese manufacturing triggered a worldwide stock hiccup that sent values plummeting. The continued plunge in world oil prices has resulted in a slowdown in exploration and, by extension, the manufacture of exploration equipment.

Read more: 2015 Salary Survey

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
The maintenance journey has been a long, slow trek for most manufacturers and has gone from preventive maintenance to predictive maintenance.
Featured articles highlight technologies that enable the Industrial Internet of Things, IIoT-related products and strategies to get data more easily to the user.
This digital report will explore several aspects of how IIoT will transform manufacturing in the coming years.
Maintenance Manager; California Oils Corp.
Associate, Electrical Engineering; Wood Harbinger
Control Systems Engineer; Robert Bosch Corp.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me