Bildunterschrift
Figure 1: Overview of our approach, ”MILD”. We train VAEs to reconstruct the observations of the interactions agents (x11:t,x21:t) with an HSMM prior to learn a joint distribution over the latent space trajectories (z11:t,z21:t) of the interacting agents. During test time, the observed agent’s latent trajectory p(z2t | z11:t) conditions the HSMM to infer the controlled agent’s latent trajectory x̂ 2t which is decoded to generate the agent’s real-world trajectory.
