Artificial intelligence (AI) is beginning to make an impact on manufacturing. Data from predictive maintenance is moving into useful analytics. The manufacturing supply chain is getting optimized. AI is helping manufacturers to improve uptime, increase yield, and reduce downtime. Recently we’ve seen machine learning bring significant benefits to manufacturing. This can come in the form of smarter robotics as well as plant and warehouse optimization.

We visited with Chris Nicholson, CEO of Pathmind, a company that applies AI to industrial operations. Nicholson explained some of the newer areas of AI, including reinforcement learning.

Reinforcement Learning in Robots

Deep reinforcement learning (RL) consists of algorithms that make it easier for manufacturers to take the leap of faith with robots from simulation to the real world. Nicholson repeated a Google statement that when they crack AI with deep RL, they will crack the whole nut.

RL is part of the larger world of AI and machine learning (ML). “Reinforcement learning is a branch of AI. AI is the big umbrella term. Machine learning is part of AI. With RL, you have deep learning and within the deep neural network of an ML learning network,” said Nicholson. “You can apply the algorithms to robots. As the robots learn their tasks, they get a numerical reward. When they screw up, they get an RL penalty. Those processes help them learn.”

The RL for the robot is a combination of learning the robot’s capabilities and determining how they can function in the robot’s environment. “What it means to teach a robot what to do is there’s a lot of intelligence in the software,” said Nicholson. “What can the arm do? Where can it move? Where can it go and how fast? The software learns these capabilities and then lets the algorithm make the choices.”

The Beginning of Embodied AI

Embodied systems perceive, act, and learn (by artificial means) to interact with their environment. “There are positive indicators that show we are on the road to making embodied AI work,” said Nicholson. “Advances in deep RL can compute on the edge so the AI training for the robots is based on simulations and historical data.”

The decisions the robot makes will be more effective if the robot doesn’t have to send signals to the cloud. So edge computing is often the best solution. “You can determine which machine bodies that AI can inhabit. Most of the time AI lives in the cloud, not in the four-sided world. With robots, it can be embodied AI at the edge,” said Nicholson. “Whatever processes the robot is capable of can be added to the robot so it is emboldened. The AI can become like a kid in a sandbox learning what it can do.”

The Growing Role of Digital Twins

AI/ML alone cannot solve the problem of a boat blocking the Suez Canal, but ML and simulation together can do so by introducing a digital twin. Digital twins are expanding the ability to respond to events not contained in historical data. They can provide examples for “what-if” questions that they can go on to answer.

The digital twin allows AI to work in a complex environment with multiple systems operating simultaneously. The digital twin can simulate changes and see the results before moving into actual operation. “Different people mean different things with digital twins. It starts as a tool for thought, a digital model. That model of the world is a complex system,” said Nicholson. “The simulation in the digital twin shows that if you change this one thing, you see what it does. It allows AI to explore a complex space. Digital twins are what-if machines. They offer the exploration of possibilities.”

The digital twin is a safe and inexpensive way to experiment with a variety of solutions to a complex problem. “A lot of what-ifs are too expensive to recreate in the physical world. You can’t just build a fulfillment center. It helps to have a digital model based on traffic and put the AI at different points to test how it will operate,” said Nicholson. “Amazon is very involved in simulation with their robots. The big MES providers like Siemens have different forms of AI going into their warehouses and exposing their robots to what-ifs through digital twins.”

AI Can Offset Climate Change

AI and ML tools are getting used to predict future energy consumption patterns in manufacturing. This mitigates soaring energy costs and also helps offset climate change. AI also helps to sort out chaotic systems such as renewables. “Training these AI models is burning tons of energy. That’s not false. It does take energy,” said Nicholson. “But what people are missing is that AI models are designed to help companies with enormous physical systems operate more efficiently.”

While AI takes up a lot of processing energy, the results in efficiency savings can far outweigh the expense in energy consumption. “AI can help us make more with less. We can cut down on waste with optimization. We can get growth without consuming more,” said Nicholson. “We can train an optimization model in 20 minutes to save a company tens of millions of dollars of energy consumption per year. The advantages can be huge. That’s already happening.”

AI can help plant managers figure out what equipment is best for what task at what time. These are issues that are not easily soloved without computer analysis. “In factories and warehouses, moving batches from station to station can be optimized by determining which machine carries the batch. When I hear about AI in factories, it’s often about a single machine and controlling a robot arm,” said Nicholson. “It’s a different scale when you have a team of machines. When you teach collective goals, one machine sacrifices its performance for the good of the whole.”

AI ultimately allows plant managers to solve complex problems efficiently. “AI creates emergent behavior. We look at complexity theory as we move into new phases,” said Nicholson. “We’re able to steer emergent behavior. That means we can master more complex problems. The amount of sophistication means we can grasp bigger problems and bend them to our will, such as efficiency.”