Adaptive Autonomous Assembly

Adaptive Autonomous Assembly

Niklas_Funk_Adaptive Autonomous Assembly_Figure1 Niklas_Funk_Adaptive Autonomous Assembly_Figure1

Figure 1: Illustrating the assembly process. In this experiment, the trained agent successfully assembles the demanded shape (illustrated in yellow), using the building blocks that are avaiable.
 

Niklas Funk
Niklas_Funk_Adaptive Autonomous Assembly_Figure2 Niklas_Funk_Adaptive Autonomous Assembly_Figure2

Figure 2: Illustrating the training curve, i.e., evolution of reward over time for different graph architectures. As can be seen in the plot, our chosen multi-head attention (MHA) graph architecture outperforms the single-head attention (SHA) one, as well as the S2V architecture.

Niklas Funk

Introduction

Autonomous robotic assembly requires a well-orchestrated sequence of high-level actions and smooth manipulation executions. Learning to assemble complex 3D structures remains a challenging problem that requires drawing connections between target designs and building blocks, and creating valid assembly sequences considering structural stability and feasibility. To address the combinatorial complexity of the assembly tasks, we propose a multi-head attention graph representation that can be trained with reinforcement learning (RL) to encode the spatial relations and provide meaningful assembly actions. Combining structured representations with model-free RL and Monte-Carlo planning allows agents to operate with various target shapes and building block types. We design a hierarchical control framework that learns to sequence the building blocks to construct arbitrary 3D designs and ensures their feasibility, as we plan the geometric execution with the robot-in-the-loop. We demonstrate the flexibility of the proposed structured representation and our algorithmic solution in a series of simulated 3D assembly tasks with robotic evaluation, which showcases our method’s ability to learn to construct stable structures with a large number of building blocks. As all of the used methods are computationally demanding, especially during training time, for this project to be successful, we required access to a HPC.

Methods

For realizing autonomous adaptive assembly, and in particular, resolving the resource assignment part of figuring out how to combine a given set of building blocks to achieve a desired overall structure, we investigated the combination of several components. On the representation level, we decided to use graph neural networks that are especially flexible when wanting to handle an arbitrary number of available blocks / target shapes and that have been successfully applied in related tasks. The representation is trained using reinforcement learning, i.e., deep q-learning, which is a model-free technique and should be well-suited for this task that includes combinatorial, as well as kinematic, and dynamic difficulties. Lastly, to increase reliability, we also investigate the addition of Monte-Carlo Tree Search (MCTS) during test time.

Results

The results of our experiment have shown that our proposed components are indeed capable of solving the challenging problem of learning to assemble arbitrary structures given a set of available building blocks. In particular, the graph-based neural network exhibits the desired properties of generalizing across target shapes and building blocks. Moreover, the chosen architecture is superior compared to alternative ones, and the training procedure through DQN is indeed effective to obtain the assembly policies. Last, adding MCTS at test time indeed improves the proposed method’s performance.

Discussion

While we were able to show very promising results in simulation environments, the main weakness of the current method is its limited applicability in more reaslistic scenarios. To overcome this weakness, further research with respect to object pose estimation and object tracking is necessary. Moreover, another limiting factor has been the robot’s limited workspace. Thus, in the future, we would like to combine our method with mobile manipulators.

Last Update

  • Last Update:

Participating Universities