Smart screens for industrial applications

Developers of smart screen HMIs can achieve industrial-grade reliability and real-time performance by using both Flash and C/C++ components. Design of a building automation system provides an example.



To evaluate the suitability of a Adobe Flash Lite based smart screen, QNX developers crated a buliding automation systerm and HMI.
To evaluate the suitability of a Adobe Flash Lite based smart screen, QNX developers crated a buliding automation systerm and HMI.

A smart screen can be defined as a human-machine interface (HMI) that uses interactive graphics displays on a touchscreen to offer users control of an underlying embedded system. Smart screens come in many shapes and sizes, and can be used for applications ranging from ticket kiosks to factory control systems.

Smart screens are becoming the preferred HMI for a variety of industrial control applications. These smart screens are essentially the same as those for consumer applications. The key differences lie in the underlying applications and the HMI design.

Where consumer applications may emphasize the “Wow!” factor—with morphing widgets and other non-essential but eye-catching features—industrial smart screens must provide clear and readily understandable interfaces. In addition, industrial HMIs must be designed to be consistent over many years and easily updatable. They must maintain a consistent look, feel, and functionality even as the systems beneath them change.

This requirement for consistency and ease of use does not imply that creating smart screens for industrial applications is less difficult than for consumer electronics. In addition to providing on-screen information and controls, smart screens may need to provide interactive access to layered diagrams, blueprints and maps, embedded videos, and other media. Above all, they must offer industrial-grade reliability.

Building smart screen HMIs with Adobe Flash Lite allows developers to take advantage of the Flash toolbox, which includes all the elements needed to build a clear, effective smart screen, and which supports the most stringent design requirements, including layout, layering, and multimedia support. Moreover, top-notch Flash designers and developers are relatively easy to find, thanks to the widespread acceptance of Flash as an Internet technology.

Unlike desktop Flash players, Adobe Flash Lite is designed specifically for resource-constrained embedded products. To evaluate the suitability of a Flash Lite smart screen implemented over an industrial application running in an embedded environment, we designed and implemented a building automation system and HMI. The HMI allows a user to operate a multi-zone control system, monitor system-wide alarms, and perform video surveillance. The system operator can touch the smart screen to zoom in on specific wings or rooms, adjust temperature or fan speeds, and control a variety of other functions.


This smart screen presents information gathered through the HMI connection to the underlying system controls.
This smart screen presents information gathered through the HMI connection to the underlying system controls.

The system sensors and controls are simulated, but they could be replaced by actual hardware in a real building. To more closely simulate a live environment, we designed the system to be fault tolerant. For example, if an error or a user intervention takes a building zone offline, the application detects the problem and reports it through the HMI.

A Freescale MPC8536E PowerQUICC III serves as the primary CPU, although the software stack can run on a number of different chips and architectures. The QNX Neutrino RTOS serves as the operating system, and the MPC8536E board support package provides device drivers that interface with the I2C and SPI hardware.

Three processes control each zone: one process for heating, ventilation, and air conditioning (HVAC); one for fan motor control; and one for the temperature and humidity sensor. At the top of the stack, an application using Adobe Flash Lite 3 provides the smart screen HMI.

Design challenges

A building automation system must support dynamic deployment. Physical factors often constrain module placement, and changes to building infrastructure may dictate system rearrangement.

To address the challenge, we separated the HVAC control tasks for each zone into independent processes. This separation allows the system to be distributed on multiple nodes across an Ethernet network using Transparent Distributed Processing (TDP). As a result, building engineers can arbitrarily rearrange the system’s zone deployment without having to reconfigure the master HMI system. Separating each zone’s tasks into different processes also provides greater fault resilience—the system can detect the problem and restart without affecting other functions.


The software architecture of the building automation system shows how three processes control each zone: one for heating, ventilation, and air conditioning (HVAC), one for fan motor control, and one for the temperature and humidity sensor.
The software architecture of the building automation system shows how three processes control each zone: one for heating, ventilation, and air conditioning (HVAC), one for fan motor control, and one for the temperature and humidity sensor.

The smart screen HMI presented another challenge. We wanted to use Adobe Flash because it simplifies the work of creating an attractive and effective user interface. However, we knew that compiled Flash binaries execute inside the Flash virtual machine, isolating the Flash HMI from the rest of the system. Flash code operating in a browser cannot manipulate the underlying PC environment, and, identically, an embedded Flash HMI does not normally have access to the embedded system’s underlying hardware resources.

The solution to this problem turned out to be straighforward—use an Action Script Extension (ASE) that allows Flash code to interface directly with native C/C++ code. We used this capability to interface the HMI to the HVAC, motor, and sensor processes. This strategy allows these processes to operate like traditional embedded software, which would interface to the sensors and controllers either by directly accessing the A/D hardware and GPIO pins, or by using I2C or SPI bus drivers to communicate with the devices.

As part of this project, we attempted to address key concerns about the suitability of Flash Lite for embedded industrial applications. These include:

  • Dynamic memory consumption . The amount of memory available to the Flash virtual machine’s heap can be completely constrained so that it will not exhaust system memory.

  • Real-time performance . Flash does not offer real-time performance. Developers can mitigate this by moving any real-time requirements into native C/C++ code and then communicating between that code and the Flash HMI.

  • The Flash single-thread model . The HMI can stall if a portion of it is waiting on an input that fails to arrive immediately. Developers can solve the problem by moving code that can block out into a C/C++ module.

Bottom line: Developers can satisfy the twofold requirements of industrial-grade reliability and real-time performance by implementing an appropriate division of the HMI between Flash and C/C++ components.

Author Information
Andy Gryc writes for QNX Software Systems, maker of middleware, development tools, realtime operating system software and services for embedded design.

The Top Plant program honors outstanding manufacturing facilities in North America. View the 2015 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
Pipe fabrication and IIoT; 2017 Product of the Year finalists
The future of electrical safety; Four keys to RPM success; Picking the right weld fume option
A new approach to the Skills Gap; Community colleges may hold the key for manufacturing; 2017 Engineering Leaders Under 40
Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
The cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Power system design for high-performance buildings; mitigating arc flash hazards
VFDs improving motion control applications; Powering automation and IIoT wirelessly; Connecting the dots
Natural gas engines; New applications for fuel cells; Large engines become more efficient; Extending boiler life

Annual Salary Survey

Before the calendar turned, 2016 already had the makings of a pivotal year for manufacturing, and for the world.

There were the big events for the year, including the United States as Partner Country at Hannover Messe in April and the 2016 International Manufacturing Technology Show in Chicago in September. There's also the matter of the U.S. presidential elections in November, which promise to shape policy in manufacturing for years to come.

But the year started with global economic turmoil, as a slowdown in Chinese manufacturing triggered a worldwide stock hiccup that sent values plummeting. The continued plunge in world oil prices has resulted in a slowdown in exploration and, by extension, the manufacture of exploration equipment.

Read more: 2015 Salary Survey

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.
The maintenance journey has been a long, slow trek for most manufacturers and has gone from preventive maintenance to predictive maintenance.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This digital report will explore several aspects of how IIoT will transform manufacturing in the coming years.
Maintenance Manager; California Oils Corp.
Associate, Electrical Engineering; Wood Harbinger
Control Systems Engineer; Robert Bosch Corp.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me