Adaptive Autonomous Assembly
Introduction
Autonomous robotic assembly requires a well-orchestrated sequence of high-level actions and smooth manipulation executions. Learning to assemble complex 3D structures remains a challenging problem that requires drawing connections between target designs and building blocks, and creating valid assembly sequences considering structural stability and feasibility. To address the combinatorial complexity of the assembly tasks, we propose a multi-head attention graph representation that can be trained with reinforcement learning (RL) to encode the spatial relations and provide meaningful assembly actions. Combining structured representations with model-free RL and Monte-Carlo planning allows agents to operate with various target shapes and building block types. We design a hierarchical control framework that learns to sequence the building blocks to construct arbitrary 3D designs and ensures their feasibility, as we plan the geometric execution with the robot-in-the-loop. We demonstrate the flexibility of the proposed structured representation and our algorithmic solution in a series of simulated 3D assembly tasks with robotic evaluation, which showcases our method’s ability to learn to construct stable structures with a large number of building blocks. As all of the used methods are computationally demanding, especially during training time, for this project to be successful, we required access to a HPC.
Methods
For realizing autonomous adaptive assembly, and in particular, resolving the resource assignment part of figuring out how to combine a given set of building blocks to achieve a desired overall structure, we investigated the combination of several components. On the representation level, we decided to use graph neural networks that are especially flexible when wanting to handle an arbitrary number of available blocks / target shapes and that have been successfully applied in related tasks. The representation is trained using reinforcement learning, i.e., deep q-learning, which is a model-free technique and should be well-suited for this task that includes combinatorial, as well as kinematic, and dynamic difficulties. Lastly, to increase reliability, we also investigate the addition of Monte-Carlo Tree Search (MCTS) during test time.
Results
The results of our experiment have shown that our proposed components are indeed capable of solving the challenging problem of learning to assemble arbitrary structures given a set of available building blocks. In particular, the graph-based neural network exhibits the desired properties of generalizing across target shapes and building blocks. Moreover, the chosen architecture is superior compared to alternative ones, and the training procedure through DQN is indeed effective to obtain the assembly policies. Last, adding MCTS at test time indeed improves the proposed method’s performance.
Discussion
While we were able to show very promising results in simulation environments, the main weakness of the current method is its limited applicability in more reaslistic scenarios. To overcome this weakness, further research with respect to object pose estimation and object tracking is necessary. Moreover, another limiting factor has been the robot’s limited workspace. Thus, in the future, we would like to combine our method with mobile manipulators.