Using a historian to store machine data

Cover story: HMI-historian integration cover story: Integration of historians with human-machine interface software allows knowledge workers to change the future based on past collection of big data. Here’s the why and how of linking your machine HMI to a historian, and the best ways to view and analyze historical data. See 8 benefits of historical machine data and 7 ways to view and analyze machine data.

By Richard Clark May 5, 2015

Historians provide a great way to store and access huge amounts of machine data, but this time-based information derived from "big data" collection must often be transformed for use by machine operators, plant engineers, and management in factories and plants. More big data value can be gained by making it more accessible for viewing and analysis, often by connecting an HMI to a historian, which is then connected to other software platforms. This allows viewing and analysis of data at the HMI, the historian, and other software platforms.

Historians can take many forms, from simple spreadsheets or personal database files, to full-featured database products available from a variety of manufacturers and running on many platforms, to dedicated data historian products capable of capturing and storing up to tens of millions of data points per second. A variety of methods are available to make information stored in historians readily actionable, such as trending, reporting, and pushing the data to external data analysis platforms.

With most machines, one or more controllers are connected to a gateway for HMI/supervisory control and data acquisition (SCADA) software; the gateway acts as a data concentrator. The HMI/SCADA gateway is in turn connected to a historian. If the HMI/SCADA gateway is PC-based, the historian may be able to reside on the same PC.

If the HMI/SCADA gateway is running on an embedded platform, the historian will probably be an external PC-based system connected to the embedded platform via Ethernet. In some cases where machine connectivity is intermittent, it may be necessary to use an embedded database product to store historical data temporarily. The data can then be replicated later to a larger database or historian when a connection is available. This is common for tank farms or remote pipelines using general packet radio service (GPRS) or satellite modem communications.

The HMI/SCADA gateway sends machine operation data to the historian for long-term data storage. Other data is often entered into the historian manually, such as fault causes, detailed analysis, and quality assurance (QA) or statistical process control (SPC) product testing results.

The gateway accesses data stored in the historian for presentation to operators and engineers. The historian can also push this data to external data warehouses for security purposes, and it can push data to various software analyses platforms, or to business intelligence and overall equipment effectiveness (OEE) dashboards.

Historical machine data value

Although all HMIs provide multiple ways to view current machine operating data, it’s often necessary to view historical machine operation data as it provides a number of benefits (Table 1).

Many changes are typically made to machine operation over the course of months or even years. Some are made in response to production demand, others are made manually in an effort to improve machine operation, and some are made automatically by the machine’s control system. Taking these changes together, it can be difficult to know exactly which machine operation parameters were in place at any given time, and how these parameters affected machine output.

Viewing this type of historical data, often via the HMI, can allow plant personnel to compare current to historical machine operation, revealing ways to improve performance. For example, analysis of production information may show that very high quality products were being produced six months ago at a level that current production can’t duplicate. Analyzing machine operating parameters can often reveal key differences, allowing operators and engineers to restore machine operation to an optimal level.

Predictive maintenance can be applied to machines by reviewing historical data and using it to adjust maintenance intervals. When a certain limit is reached, a certain time has elapsed, or certain measurements have been exceeded, the machine or process should be serviced to bring it back into operational specifications, or parts should be replaced. 


There are many ways to use historical data for statistical analysis. A very common way is to use an OEE display driven off the production data. The OEE indicates how well the plant or machine is operating, and can help pinpoint operational and production issues that might otherwise be overlooked.

Another way is to use a business intelligence dashboard showing a more detailed view of the OEE data, which can be used to quickly pinpoint localized problems in a machine or process when compared to historical views (Figure 2).

Historical machine data also is used in enterprise resource planning (ERP) integration to make business decisions based on intelligent plant data. These business decisions might be as simple as ordering raw materials based on production demand, or scheduling main and backup suppliers. In addition, downstream uses of this data might inform or modulate the production line to not overrun the supply chain.

Historical data that ERP integration can provide also may be used to schedule ahead and create contracts for lower cost raw materials. The materials can be purchased based upon a historical need, rather at current market price. Regulatory agencies also may require production data and batch information to be stored for long periods of time in case of recall or other quality needs, as in the case of facilities required to comply with U.S. Food and Drug Administration (FDA) 21 CFR Part 11 and similar regulations. Purpose-built historians provide a foolproof method of saving and recalling data for these and other purposes.

Access to historical machine operation data can provide real-time benefits, but how can data be transferred between an HMI and a historian?

Going native

Linking an HMI to a historian for two-way information transfers isn’t always an easy task, despite the preponderance of open systems. While it’s possible in theory to link most any HMI to any historian, integration is not always easy in practice.

Both the HMI and the historian will most likely be PC-based, so a physical Ethernet connection will be easy to establish, along with a common Ethernet protocol. But linking the HMI data to specific historians can be an involved process, often consuming many human hours.

For this reason, many users prefer native solutions where the HMI and the historian are purchased from the same company. These solutions will be easier to implement than integrating products from two or more suppliers. [Parallel data structures also can help.]

For example, several HMI packages designed with the ability to manage and control many thousands of data points can easily and efficiently save that historical data several times per second to a historian, if needed. Once the HMI-historian integration challenge is solved, plant personnel can view and analyze historical data in a variety of ways (Table 2).

Trends and reports

Most every HMI product provides the ability to easily create trend screens to view data, and once the HMI is linked to the historian it’s a simple matter to view and compare current to historical data on a trend screen. Viewing data in this fashion can often reveal areas for improvement. Many historians also allow easy creation of trend screens, providing another platform for viewing and analyzing data.

Similarly, most HMI and historian products provide the means to create a variety of reports. Historical data reports are required in implementations where batch information is paramount. These sorts of implementations are used in many life sciences, food, and beverage manufacturing facilities.

Long reports of analog data are, generally speaking, not very useful except perhaps in troubleshooting or when used with historical trending at-a-glance. However, batch reporting is almost always a requirement.

Therefore, reporting can take on a number of forms (Figure 3), and reports are usually saved as Web pages or Adobe PDF documents to preserve the integrity of the report across a wide variety of display platforms from large screens to handheld devices.

These reports are often used by management to get information needed at a glance.

Effective data transfer

Once machine and process data has been saved in a historian, it often needs to be transferred to the areas of the business that need it. Many times, the sheer amount of data in the historian may overwhelm the network connecting the historian to the business systems.

To reduce the traffic, most production historians simply push or replicate a data subset to one or more database servers or files that are made available for transfer to business systems. These replications are usually in the form of a stored procedure as one or more SQL queries, which can be exported to a variety of programs, such as spreadsheets, databases, or data analysis platforms.

These external platforms allow users to analyze data supplied to the historian by the HMI in a manner suiting their purposes, using familiar platforms. For example, many use spreadsheets to perform data analysis, as most are very familiar with these platforms. Others may use databases in their daily work and would prefer to use these tools for data analysis. Still others may require very sophisticated data analysis platforms dedicated to one purpose, such as SPC/statistical quality control (SQC) efforts.

Whatever the data viewing and analysis requirements, they can usually be met by connecting an HMI to a historian, which is in turn connected to other software platforms. This allows viewing and analysis of data at the HMI, the historian, and other software platforms.

– Richard Clark is automation and process control engineer, Wonderware by Schneider Electric; edited by Mark T. Hoske, content manager, CFE Media, Control Engineering,

Key concepts

  • Big data analysis and viewing can help by connecting human-machine interface (HMI) and historian software.
  • Historical data is more useful when it can influence events in real time. 
  • Using software from one company can simplify integration.

Consider this

What in your future might you change with real-time reminders of the past? What would that be worth? 

ONLINE extra 

This online portion of the May Control Engineering print and digital edition cover story includes more information and 3 links below for more HMI-historian integration help.

Original content can be found at Oil and Gas Engineering.