Smart screens for industrial applications

Developers of smart screen HMIs can achieve industrial-grade reliability and real-time performance by using both Flash and C/C++ components. Design of a building automation system provides an example.



To evaluate the suitability of a Adobe Flash Lite based smart screen, QNX developers crated a buliding automation systerm and HMI.

To evaluate the suitability of a Adobe Flash Lite based smart screen, QNX developers crated a buliding automation systerm and HMI.

A smart screen can be defined as a human-machine interface (HMI) that uses interactive graphics displays on a touchscreen to offer users control of an underlying embedded system. Smart screens come in many shapes and sizes, and can be used for applications ranging from ticket kiosks to factory control systems.

Smart screens are becoming the preferred HMI for a variety of industrial control applications. These smart screens are essentially the same as those for consumer applications. The key differences lie in the underlying applications and the HMI design.

Where consumer applications may emphasize the “Wow!” factor—with morphing widgets and other non-essential but eye-catching features—industrial smart screens must provide clear and readily understandable interfaces. In addition, industrial HMIs must be designed to be consistent over many years and easily updatable. They must maintain a consistent look, feel, and functionality even as the systems beneath them change.

This requirement for consistency and ease of use does not imply that creating smart screens for industrial applications is less difficult than for consumer electronics. In addition to providing on-screen information and controls, smart screens may need to provide interactive access to layered diagrams, blueprints and maps, embedded videos, and other media. Above all, they must offer industrial-grade reliability.

Building smart screen HMIs with Adobe Flash Lite allows developers to take advantage of the Flash toolbox, which includes all the elements needed to build a clear, effective smart screen, and which supports the most stringent design requirements, including layout, layering, and multimedia support. Moreover, top-notch Flash designers and developers are relatively easy to find, thanks to the widespread acceptance of Flash as an Internet technology.

Unlike desktop Flash players, Adobe Flash Lite is designed specifically for resource-constrained embedded products. To evaluate the suitability of a Flash Lite smart screen implemented over an industrial application running in an embedded environment, we designed and implemented a building automation system and HMI. The HMI allows a user to operate a multi-zone control system, monitor system-wide alarms, and perform video surveillance. The system operator can touch the smart screen to zoom in on specific wings or rooms, adjust temperature or fan speeds, and control a variety of other functions.


This smart screen presents information gathered through the HMI connection to the underlying system controls.

This smart screen presents information gathered through the HMI connection to the underlying system controls.

The system sensors and controls are simulated, but they could be replaced by actual hardware in a real building. To more closely simulate a live environment, we designed the system to be fault tolerant. For example, if an error or a user intervention takes a building zone offline, the application detects the problem and reports it through the HMI.

A Freescale MPC8536E PowerQUICC III serves as the primary CPU, although the software stack can run on a number of different chips and architectures. The QNX Neutrino RTOS serves as the operating system, and the MPC8536E board support package provides device drivers that interface with the I2C and SPI hardware.

Three processes control each zone: one process for heating, ventilation, and air conditioning (HVAC); one for fan motor control; and one for the temperature and humidity sensor. At the top of the stack, an application using Adobe Flash Lite 3 provides the smart screen HMI.

Design challenges

A building automation system must support dynamic deployment. Physical factors often constrain module placement, and changes to building infrastructure may dictate system rearrangement.

To address the challenge, we separated the HVAC control tasks for each zone into independent processes. This separation allows the system to be distributed on multiple nodes across an Ethernet network using Transparent Distributed Processing (TDP). As a result, building engineers can arbitrarily rearrange the system’s zone deployment without having to reconfigure the master HMI system. Separating each zone’s tasks into different processes also provides greater fault resilience—the system can detect the problem and restart without affecting other functions.


The software architecture of the building automation system shows how three processes control each zone: one for heating, ventilation, and air conditioning (HVAC), one for fan motor control, and one for the temperature and humidity sensor.

The software architecture of the building automation system shows how three processes control each zone: one for heating, ventilation, and air conditioning (HVAC), one for fan motor control, and one for the temperature and humidity sensor.

The smart screen HMI presented another challenge. We wanted to use Adobe Flash because it simplifies the work of creating an attractive and effective user interface. However, we knew that compiled Flash binaries execute inside the Flash virtual machine, isolating the Flash HMI from the rest of the system. Flash code operating in a browser cannot manipulate the underlying PC environment, and, identically, an embedded Flash HMI does not normally have access to the embedded system’s underlying hardware resources.

The solution to this problem turned out to be straighforward—use an Action Script Extension (ASE) that allows Flash code to interface directly with native C/C++ code. We used this capability to interface the HMI to the HVAC, motor, and sensor processes. This strategy allows these processes to operate like traditional embedded software, which would interface to the sensors and controllers either by directly accessing the A/D hardware and GPIO pins, or by using I2C or SPI bus drivers to communicate with the devices.

As part of this project, we attempted to address key concerns about the suitability of Flash Lite for embedded industrial applications. These include:

  • Dynamic memory consumption . The amount of memory available to the Flash virtual machine’s heap can be completely constrained so that it will not exhaust system memory.

  • Real-time performance . Flash does not offer real-time performance. Developers can mitigate this by moving any real-time requirements into native C/C++ code and then communicating between that code and the Flash HMI.

  • The Flash single-thread model . The HMI can stall if a portion of it is waiting on an input that fails to arrive immediately. Developers can solve the problem by moving code that can block out into a C/C++ module.

Bottom line: Developers can satisfy the twofold requirements of industrial-grade reliability and real-time performance by implementing an appropriate division of the HMI between Flash and C/C++ components.

Author Information

Andy Gryc writes for QNX Software Systems, maker of middleware, development tools, realtime operating system software and services for embedded design.

No comments
The Top Plant program honors outstanding manufacturing facilities in North America. View the 2013 Top Plant.
The Product of the Year program recognizes products newly released in the manufacturing industries.
The Leaders Under 40 program features outstanding young people who are making a difference in manufacturing. View the 2013 Leaders here.
The new control room: It's got all the bells and whistles - and alarms, too; Remote maintenance; Specifying VFDs
2014 forecast issue: To serve and to manufacture - Veterans will bring skill and discipline to the plant floor if we can find a way to get them there.
2013 Top Plant: Lincoln Electric Company, Cleveland, Ohio
Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Plant Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.

Bring focus to PLC programming: 5 things to avoid in putting your system together; Managing the DCS upgrade; PLM upgrade: a step-by-step approach
Balancing the bagging triangle; PID tuning improves process efficiency; Standardizing control room HMIs
Commissioning electrical systems in mission critical facilities; Anticipating the Smart Grid; Mitigating arc flash hazards in medium-voltage switchgear; Comparing generator sizing software

Annual Salary Survey

Participate in the 2013 Salary Survey

In a year when manufacturing continued to lead the economic rebound, it makes sense that plant manager bonuses rebounded. Plant Engineering’s annual Salary Survey shows both wages and bonuses rose in 2012 after a retreat the year before.

Average salary across all job titles for plant floor management rose 3.5% to $95,446, and bonus compensation jumped to $15,162, a 4.2% increase from the 2010 level and double the 2011 total, which showed a sharp drop in bonus.

2012 Salary Survey Analysis

2012 Salary Survey Results

Maintenance and reliability tips and best practices from the maintenance and reliability coaches at Allied Reliability Group.
The One Voice for Manufacturing blog reports on federal public policy issues impacting the manufacturing sector. One Voice is a joint effort by the National Tooling and Machining...
The Society for Maintenance and Reliability Professionals an organization devoted...
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.
Maintenance is not optional in manufacturing. It’s a profit center, driving productivity and uptime while reducing overall repair costs.
The Lachance on CMMS blog is about current maintenance topics. Blogger Paul Lachance is president and chief technology officer for Smartware Group.