Vision guided robotics: 'Seeing’ is believing

Because heavy lifting and fast repetitive motions create a hostile environment for people, robots make sense for many manufacturing tasks. But how do you combine the speed, accuracy and repeatability of a robot with the adaptability of a human? Guide it with machine vision.

Vision and robotics first converged when manufacturers discovered that many robot applications lacked the flexibility to easily accommodate smaller batches and frequent changeovers required for mixed-model processing.

Unlike blind robots, vision-guided robots don’t depend on costly precision fixtures to hold parts, require additional labor to load and orient parts or need upstream actuators, sorters and feeders to separate parts for processing. Consequently, vision guided robotics (VGR) allows manufacturers to more easily process various part types without tooling changeovers. VGR provides the added benefit of automatic collision avoidance for safer work cells.

In addition to providing manufacturing flexibility to accommodate product changes, today’s vision systems are less expensive, making many new VGR applications cost justifiable. Simplified calibration, easier integration and new connectivity standards make VGR faster and easier to deploy. Beyond locating parts for pick-and-place or guiding a robot to assemble components, machine vision also can inspect, measure and read linear barcodes and data matrix codes as products are being handled or assembled.

VGR implementations vary depending on the application, but most use image analysis software to calculate positional information from a 2D image and provide it to a robot controller. Pick-and-place applications generally use a camera to acquire images of an area on a conveyor carrying objects for packaging, palletizing or assembly. The vision system finds and computes the location of the objects on the conveyor, then converts the location into X-Y and theta coordinates, which it reports to the robot.

Other VGR applications fall somewhere between 2D and 3D data gathering. These typically use apparent changes in perspective or size to calculate 3D data. Layered bin picking, for example, presents objects in random positions and orientations in a stack of trays. When a camera views a stack of parts, the top object appears smaller as the stack gets shorter and its distance from the camera increases.

In these applications, the vision system uses the apparent change in size to calculate the top tray’s height so that the robot can continue to add or remove parts from the stack. In the most complex applications, multiple cameras or structured light techniques are used to provide 3D data.

In both 2D and 3D VGR applications, calibrating the vision system’s pixel-based coordinate system with the robot’s coordinate system is vital for success. Whether the application involves conveyor tracking for pick-and-place, palletizing or component assembly, vision-to-robot calibration is required to maintain system accuracy and repeatability.

Calibration is one of the biggest challenges because it involves more than coming up with a scaling factor that relates pixels to a measured dimension. If there’s optical distortion from the lens, or perspective changes due to the camera mounting angle, the vision software must include special algorithms to correct for these image distortions.

Standard practices have evolved to make calibration easier. The most advanced vision software now incorporates step-by-step wizard functionality to guide users through the process of correlating image pixels to robot coordinates using a variety of techniques including a grid of dots, checkerboard or custom calibration plates.

The latest software also supports multi-pose 2D calibration to optimize system accuracy and enable the use of a smaller, more manageable calibration plate in large field-of-view applications.

VGR performance is significantly limited when a vision sensor can’t provide repeatable part location due to part variability. All manufacturing processes have some variability. Experience in a wide range of industries shows that variations usually fall into several categories:

• Part rotation caused by lack of fixturing, vibration/motion on the line, etc.
• Changes in the scale of a part due to variations in the vision camera’s optical settings
• Inconsistent or poor lighting
• Parts produced in distinctly different colors, textures, shapes and sizes
• Variations caused by modifications to the production process
• Substitutions of components or materials
• Different suppliers for a single part
• The presence of oil, paint, cleaning solvents and other substances that might obscure a part or change its appearance. These may be accidentally introduced, or may be a known result of the production process.

It is important to become familiar with some of the factors that may cause parts to vary in appearance from one to another. If part variability is an issue, be sure the specified vision software supports sophisticated geometric pattern matching to ensure accurate and consistent part location.

Communications is another critical factor to consider. The easier it is to configure tight, seamless communications between the vision and robot controllers, the faster the application can be deployed. In many cases, robot manufacturers use proprietary bus architectures that complicate communications setup and use. To address this issue, it’s important that the vision software include robot drivers, sample code and other tools that make it easy to properly format communication ports and data strings with minimal effort.

Such tools make it easy for robotic systems integrators to set up communications between the vision system and a wide variety of robots without complex programming. However, the vision system should also include a software development kit that allows integrators to leverage their industry and application knowledge to develop customer or application-specific VGR and robotic inspection solutions that are tightly integrated with existing HMI and SCADA systems at a given factory.

Before specifying a vision system for robot guidance, it’s important to evaluate and understand the application thoroughly and to define the performance requirements. The performance of a vision application is typically defined by a number of attributes, which include accuracy, repeatability, precision, robustness and throughput.

For simple VGR applications, a VGR package from a robot supplier can reduce integration time. However, most real-world VGR applications are very challenging. Success depends on not only selecting the right robot, but also choosing the right hardware for the vision task. Typically, companies experienced in industrial machine vision provide the broadest range of machine vision technology, with the most accurate and reliable vision tools for part location, inspection, measurement and code reading.

Core competencies in image formation, image processing and image analysis give companies with a machine vision background a significant edge over robot vendors in delivering high-performance VGR solutions that really work.