Make better, strategic decisions in less time with Big Data

Advanced analytic solutions, including systems that mimic human decision-making, are helping oil and gas companies adapt to a world of lower prices.

By Sidney Hill, Jr. October 7, 2016

Just looking at the numbers—prices dropping as much as 70%, more than 150,000 jobs lost, and countless small- and medium-sized operators forced out of business over the past two years—you might see a bleak future for the oil and gas industry.

But there’s one irrefutable fact overshadowing any negative forecasts for this industry: the industrialized world simply cannot function without its products. That means there’s still money to be made producing oil and gas at a reasonable cost.

While the definition of "reasonable cost" may never again be the same as it was 2 years ago—when a barrel of oil was selling for more than $100—some operators are seeing the potential for turning a profit as prices have begun hovering in the $45- to $50-a-barrel range. And some industry analysts have compiled their own numbers that indicate oil and gas companies are starting to view Big Data and advanced analytics as essential tools for ensuring future profitability, no matter where prices go.

According to a report published by Technavio, Global Big Data Market in the Oil and Gas Sector 2016-2020, the market research firm sees the market for Big Data software and services growing at a compound annual rate of 30.67% for the 5-year period starting in 2015, reaching $5.41 billion in 2020. 

Big Data’s growth within the oil and gas industry

The Technavio report also states that Big Data in the oil and gas industry will grow at a tremendous rate because of its many benefits, which include the detection of faulty equipment through sensors; better drilling and connections of new wells; and a predictable approach for the maintenance of pipelines and other equipment. In addition, oil and gas companies with Big Data solutions are more efficient in managing all sorts of data, such as drilling logs, frack performance data, and production rates, which helps in optimizing well design and production.

Many industry observers say oil and gas producers initially were hesitant to adopt Big Data—despite these very real benefits—until the steep, and now persistent, dip in prices forced them to look for ways of operating more efficiently.

"We really did start seeing a shift in thinking around the end of 2014," says Charles Karren, director of oil and gas transformation strategy for Oracle, a Big Data technology supplier. "The simple reason is you can make mistakes when oil is $110 a barrel, but not when its $40 a barrel."

If prices stabilize around $50 a barrel, Karren believes companies can make realistic budgets, though they still will have little margin for error. "At this price level, any incremental gains in efficiency are a plus," he says, "shaving a few dollars in operating costs here and there can make a real difference." 

Oil and gas companies are reaching for the cloud

As companies seek those incremental gains, Karren sees them increasingly turning to two types of technology: Big Data and cloud-based analytics applications.

Using cloud-based applications might appear to be an atypical move for oil and gas companies, given their historical preference for developing their own proprietary technology platforms, largely out of a desire to protect data that might give competitors insight into their exploration and production strategies. However, Karren says information technology vendors’ track record of keeping data secure is raising the industry’s comfort level with cloud-based solutions. It also helps, particularly in the current environment, that cloud-based solutions typically cost less to implement and maintain than solutions built to operate in-house.

Big Data, however, is technology that is a more natural fit for an industry that has always revolved around analysis of large amounts of complex information.

"Oil and gas could be considered the original Big Data industry," Karren says. "Historically, when drilling wells, you never saw hydrocarbon until it came out of the ground. So, data analysis was necessary."

While analyzing complex data has always been an inherent part of finding and producing oil and gas, in recent years, a number of factors have combined to make those tasks even tougher. One of those factors is the rapid pace at which companies are now able to drill wells.

"Companies used to drill one well at a time, and it would take about 30 days," said Karren. "Now, it’s become almost a manufacturing process. Drilling a well takes 6 or 7 days, and you’re drilling multiple wells on the same pad."

With nearly the entire industry operating on this expedited timeline, there’s a constant race to get as much product out of each well as quickly as possible. That has led to what Karren calls, "an explosion" of sensors pulling data off those wells. "On any given well, especially in a deep water production site, you can have 30,000 sensors generating time-stamped data," said Karren. The manner in which that data is used also has changed.

Getting a 360-degree view of operations with Big Data

When sensors first started populating oilfields, companies would view the data they generated in real-time, use it to make any adjustments needed to keep production on track, and then discard or simply forget about the data. Now, with the constant pressure to pull product out of the ground faster and at a lower cost, companies are holding on to that data for more thorough analysis in hopes it can help in developing strategies for continuous improvement.

"They’re looking to integrate multiple data sources," said Karren, "to get a 360-degree view of their operations."

The quest for that 360-degree view has companies adding new forms of technology across the value chain. As a result, the digital oilfield has evolved to become part of the Industrial Internet of Things (IIoT). In addition to pulling data off sensors attached to production equipment, oil and gas information technology (IT) networks now include applications that feed pictures of seismic and geologic conditions, sometimes in 3-D, to users on PCs or mobile devices.

The need to add images and other forms of nontraditional data to their analysis is another reason oil and gas companies are attracted to the current generation of Big Data technology.

"What the new technology brings to the table is the ability to deal with multiple data sources," said Dave Womack, director of chemicals and petroleum for IBM.

Those sources include data coming from process automation equipment—such as time, temperature, and motion readings—that can be stored in conventional databases, as well as information such as sounds, photos, and video collected by newer sensor technologies. The former type of data is referred to as structured data; the latter type-which is becoming more prevalent in oil and gas operations-is classified as unstructured data.

According to Womack, the current generation of Big Data technology can take both types of data and format it so further intelligence can be collected and analyzed.

Jim Vick, senior vice president of information systems for Southwestern Energy, said a 360-degree view entails being able to track all of the expenses and revenue throughout the entire lifecycle of a well—the "Holy Grail" for oil and gas companies.

Southwestern Energy believes it has come close to creating its own 360-degree view via a business intelligence platform it created. The platform combines operational and financial data to populate 16 dashboards and more than 200 interactive reports that are accessed regularly by individuals across the business.

Using Big Data to make strategic business decisions

One dashboard enables tracking of all aspects of land management, maintenance, and production; while another helps the company make better decisions about drilling schedules. Doug Van Slambrouck, senior vice president of Fayetteville Shale at Southwestern Energy Company, said this platform lets Southwestern Energy accurately assess how it performed in the past as well as how it’s performing in the present-and then use that information to make better informed decisions about what it should do in the future.

As an example, he said, the platform allows for assessing the value of all of the company’s leases and determining which ones should be drilled on first and which ones can be sold to another company at a profit.

According to Karren, Southwestern Energy is not the only oil and gas company using Big Data and advanced analytics for strategic purposes. However, Karren is also quick to point out that the people using these systems are still making the actual business decisions.

"You always need that human component," said Karren. "People are making decisions based on the data being presented to them."

IBM has been working to harness that human element for corporations in various industries, including oil and gas, through a series of cognitive systems.

Cognitive systems solve problems by learning how experts in a particular field would respond to a given set of circumstances. The long-range goal is for these systems to be able to "converse" with individuals in various professions to help them reach proper decisions faster.

Womack said IBM has begun that journey with several oil and gas companies, including Woodside, the largest independent oil and gas company in Australia.

Woodside has incorporated a cognitive system into a cloud-based platform. The platform is described as a "cognitive advisory service" that Woodside’s engineering team taps into for help in finding quick resolutions to complex problems.

The engineers trust the systems’ answers because they know its core base of knowledge is built on decisions they or their peers have made in similar situations. The system developed that core knowledge by taking in more than 20 years’ worth of data that was stored across Woodside’s IT networks and analyzing actions that were taken based on that data. 

Meeting oil and gas challenges with predictive analytics

Woodside began building this platform in response to a recurring condition that had the potential to cause lengthy production shutdowns. The condition was difficult to foresee because there were no obvious data points that could indicate its imminent occurrence. Instead, engineers were relying on traditional data points, such as pressure measurements, that were only tangentially related to this condition and thus not very accurate in predicting its actual occurrence.

After studying the problem, Woodside’s engineering team discovered there was a lot of data available that could help make those predictions more accurate, but the data was so voluminous—and spread so far across the business—that it could only be harnessed through the use of Big Data and advanced analytics technology.

Once the analytics platform was installed, Woodside began using it not just to analyze existing data, but also to collect and examine newly created data in hopes of improving the accuracy of its predictions. That has resulted in establishment of data that grows by roughly 10 gigabytes per day.

What Woodside is doing with its platform is referred to as "predictive analytics," and it’s generally considered the cutting-edge for Big Data and advanced analytics in the oil and gas industry.

Womack believes these systems will help oil and gas companies successfully adapt to a world of lower prices, as well as respond to other challenges such as the lost expertise as more people leave the industry either through attrition or retirement. Industry leaders believe Big Data platforms built around cognitive systems—with the ability to capture knowledge as experts retire—will have a prominent role in the industry’s future.

Sidney Hill Jr. is a graduate from the Medill School of Journalism at Northwestern University. He has been writing about the convergence of business and technology for more than 20 years.

Take Action

Problem: Companies wanting to adapt to a world of persistent low prices need to analyze a larger amount and wider variety of data in a shorter amount of time. This fast data analysis is necessary to make better decisions in a market that exerts constant pressure to get more product off the ground faster and at a lower overall cost.

Solution: Big Data and advanced analytics solutions provide the means for storing and organizing large volumes of data from multiple sources and converting that data into a form from which users can derive the intelligence necessary to make sound business decisions.

Action to take: Conduct research on Big Data technologies and vendors who serve the oil and gas industry. Market research companies and consulting firms would be good places to start.

Original content can be found at Oil and Gas Engineering.