A Novel Unsupervised domain adaptation method for inertia-Trajectory translation of in-air handwriting

• DOI: 10.1016/j.patcog.2021.107939
• Publication Date: 2021-03-20T17:09:43Z
• Authors: Songbin Xu; Yang Xue; Xin Zhang; Lianwen Jin
• Keywords: 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology

Abstract As a new method of human-computer interaction, inertial sensor-based in-air handwriting can provide natural and unconstrained interaction to express more complex and rich information in 3D space. However, most of the existing literature is mainly focused on in-air handwriting recognition (IAHR), which makes these works suffer from the poor readability of inertial signals and the lack of labeled samples. To address these two problems, we use an unsupervised domain adaptation method to recover the trajectory of inertial signals and generate inertial samples using handwritten trajectories. In this paper, we propose an Air-Writing Translator model to learn the bi-directional translation between trajectory domain and inertial domain in the absence of paired inertial and trajectory samples. Through latent-level adversarial learning and latent classification loss, the proposed model learns to extract domain-invariant features between the inertial signal and the trajectory while preserving semantic consistency during the translation across the two domains. In addition, the proposed framework can accept inputs of arbitrary length and translate between different sampling rates. Experiments on two public datasets, 6DMG (in-air handwriting dataset) and CT (handwritten trajectory dataset), are conducted and the results demonstrate that the proposed model can achieve reliable translation between inertial domain and trajectory domain. Empirically, our method also yields the best results in comparison to the state-of-the-art methods for IAHR.