Augmented Reality for Maintenance Tasks with ChatGPT for Automated Text-to-Action

• DOI: 10.48550/arxiv.2307.03351
• Publication Date: 2023-01-01
• Authors: Fang Xu; Tri Nguyen; Jing Du
• Keywords: Memorization

Advancements in sensor technology, artificial intelligence (AI), and augmented reality (AR) have unlocked opportunities across various domains. AR large language models like GPT witnessed substantial progress are increasingly being employed diverse fields. One such promising application is operations maintenance (O&M). O&M tasks often involve complex procedures sequences that can be challenging to memorize execute correctly, particularly for novices or under high-stress situations. By marrying the advantages of superimposing virtual objects onto physical world, generating human-like text using GPT, we revolutionize operations. This study introduces a system combines AR, Optical Character Recognition (OCR), model optimize user performance while offering trustworthy interactions alleviating workload tasks. provides an interactive environment controlled by Unity game engine, facilitating seamless interaction between realities. A case (N=15) conducted illustrate findings answer research questions. The results indicate users complete similarly less time our proposed AI system. Moreover, collected data also suggests reduction cognitive load increase trust when executing same