Physical AI insights

• Physical AI can remove barriers that currently exist to fully automating manufacturing processes.
• Software can program robots to automate processes.
• AI will enable robot manufacturers to build robots that are intuitive, user-friendly and adaptable to diverse production environments.

Industrial and manufacturing plants are entering an era of advanced, artificial intelligence (AI)-boosted robotics where robots can learn from and adapt to their surroundings in real-time. This opens new possibilities for complex collaborative tasks, transforming robots from tools into intelligent partners.

Physical AI development – the use of AI and machine learning technologies with robotics – will change the way people program robots. AI will handle the line-by-line programming of motions and inputs/outputs, allowing users to command higher-level robot behaviors to complete tasks.

Imitation/reinforcement learning, vision language action models and robotics foundation models are promising research areas for physical AI. This research will simplify how we control and interact with robots and create solutions to tasks that are currently difficult to achieve with robots, such as handling textiles and connecting flexible cables.

Figure 1: Universal Robots launched the AI Accelerator. The toolkit is enabled by UR’s next-generation software platform PolyScope X and is powered by NVIDIA Isaac accelerated libraries and AI models, running on the NVIDIA Jetson AGX Orin system-on-module. The toolkit also includes the Orbbec Gemini 335Lg 3D camera. Depicted here from the ROSCon exhibition 2024 is a demo of the AI Accelerator with a computer numerical control machine tending application using a UR5e. Courtesy: Universal Robots

How physical AI can assist with existing applications

If you’re a manufacturer who has just begun a collaborative automation journey, it might be hard to see how AI can help near-term. But AI allows cobot makers to merge the flexibility of AI with mature programming paradigms. The end-result is a range of solutions that can handle workspace variance and provide the speed and precision that industrial customers expect.

AI will largely eliminate the need for experts when deploying robots. We will still need robotics engineers, integrators and other skilled experts in the future, but there can’t be an expert on every factory floor. AI will remove some of the hurdles around robotics expertise, accelerating the introduction of robots in the process.

Meanwhile, generative AI can help create standardized solutions. The challenges faced by the automation industry are similar in many companies. Generative AI allows industry to standardize both problems and solutions, creating more reusable robot behaviors. This will eliminate the need to reinvent the wheel every time a new robot is installed.

AI also boosts robots’ ability in unpredictable and dynamic environments. Vision technology with real-time feedback from 3D cameras is a huge enabler of autonomous navigation and obstacle detection. This creates possibilities for introducing robots outside of the structured environment of a factory floor. One example is in construction where robots must handle project variations while working side-by-side with workers.

Deep-learning AI vision’s role in physical AI

Machine vision is widely deployed in manufacturing environments, primarily for locating and inspecting parts. However, fewer than 20% of today’s cobot applications use vision systems. That’s because most non-AI vision systems require expert configuration, making them complex and costly to program and maintain. As a result, most industrial users prefer to spend time and money securing their parts rather than benefiting from the increased flexibility that vision enables.

When a person sees a familiar object, they can still recognize it when it has a different surface finish, under different lighting conditions, against a different background or even when it’s a slightly different size and shape. This level of visual intelligence is hard for traditional vision systems that rely heavily on high contrast between background and object and repeatable size and shape of object to detect it.

Deep-learning vision systems allow us to train this kind of variability into a single model so it can handle myriad environmental variations. It also means it’s no longer necessary for a user to obtain hundreds or thousands of images to train a model. A wide range of off-the-shelf, pre-trained models are available with permissive licensing that can be repurposed for a wide range of industrial tasks with only a short retraining process that typically requires just 50 images that can be automatically segmented and labelled.

Figure 2: Universal Robots previewed the AI Accelerator at IMTS 2024 in Chicago. Courtesy: Universal Robots

Physical AI hardware and software perceptions

NVIDIA’s Isaac ROS and Isaac manipulator make it easy to bring advanced perception capabilities into existing robot programs. Use cases include:

• Object detection: Used to locate and pick up objects in the robot workspace, reducing the need for rigid mechanical fixtures.
• Workspace check: An inspection capability used to find out “Is this thing in the workspace in the state that it should be for the robot to complete its task?” In a computer numerical control machine-tending application, for example, this capability ensures the work holding is clear and the tools in the machine are intact and clean.
• Workspace realignment: It makes sense to move robots around production environments to complete different tasks at different times. However, realigning to a workspace can be difficult because it requires precise placement of the robot and/or reteaching all the coordinate frames. A camera at the end of a robot arm automates this process so the robot can check where it is relative to the rest of the workspace and carry on with minimal fuss.
• Path planning: Plotting waypoints across the workspace to produce the optimal trajectory for a robot to get around and in and out of machines can be tricky, especially for end users new to automation. Automatic path planning makes this process much easier. But difficulties in providing the path planner with a detailed model of the robot environment is an obstacle to widespread adoption. AI can also help here, and new functionality is expected in 2025 to deal with this issue.

Ways physical AI can assist in complex warehouse environments

Complex warehouse requirements and dynamic environments can benefit from using 3D machine vision with AI to precisely identify, pick up and deliver pallets. The MiR1200 Pallet Jack solution, for example, is trained on more than 1.2 million real and synthetic images and combines data from four red, green, blue and depth (RGBD) cameras to enable fast and precise pallet handling.

The solution combines MiR’s autonomous mobile robot hardware with the NVIDIA Jetson platform. Jetson provides the advanced computing required to handle so much data in real time. By fusing feedback from RGBD cameras and 3D light detection and ranging, the robot can detect obstacles in 3D space for fully autonomous navigation.

Many times, when automated cells fail unexpectedly, the issues are caused by the world around the robot not being fixed in place well enough for the robot to do its job. It is not the robot’s fault, but traditional automation is extremely reliant on things remaining exactly as they were when their program was created. AI enables greater flexibility and adaptability, creating an opportunity to make cobots more flexible and to prevent many issues from ever occurring.

Many businesses, particularly small- and medium-sized manufacturers, face barriers such as limited resources and technical expertise. AI allows robot manufacturers to build robots that are even more intuitive, user-friendly and adaptable to diverse production environments.

By prioritizing simplicity, reliability and real-world functionality, manufacturers can introduce AI without the steep learning curve or infrastructure overhaul often associated with new technologies.

Leveling up in automation

Some cobot makers have barely incorporated AI into their offerings. But even companies that have begun providing AI capabilities still have a considerable way to go. Comparisons with the development of self-driving cars are useful, particularly the five stages that describe the transition from manual driving to fully autonomous driving.

Currently, self-driving car industry isn’t on level 5, (full driving automation) but there are a lot of level 2 (partial driving automation), 3 (conditional driving automation) or 4 (high driving automation) technologies in various stages of development, such as adaptive cruise control, which has turned a manual process into a semi-automated process, making driving smoother, easier and safer.

The same goes for industrial robots, including cobots. AI will one day lead to level 5 robots that can think and figure out how to solve problems by themselves without prior programming. That day has not arrived, but we are already seeing plenty of breakthroughs on levels 2, 3 and 4 that provide real value to businesses.

One example from logistics is a new solution that allows cobots to perform order picking autonomously. When compared to manual processes, this significantly enhances the speed and accuracy of order fulfilment in warehouses and logistics centers.

That example is not level 5, but it is an intermediate-stage technological innovation that is already delivering a lot of value to industry.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our WTWH Media editorial team and getting the recognition you and your company deserve. Click here to start this process.