Speech Translation with Large Language Models: An Industrial Practice

• DOI: 10.48550/arxiv.2312.13585
• Publication Date: 2023-01-01
• Authors: Zhichao Huang; Rong Ye; Tom Ko; Qianqian Dong; Shanbo Cheng; Mingxuan Wang; Hang Li
• Keywords: Benchmark (surveying); Speech translation

Given the great success of large language models (LLMs) across various tasks, in this paper, we introduce LLM-ST, a novel and effective speech translation model constructed upon pre-trained LLM. By integrating (LLM) with encoder employing multi-task instruction tuning, LLM-ST can produce accurate timestamped transcriptions translations, even from long audio inputs. Furthermore, our findings indicate that implementation Chain-of-Thought (CoT) prompting yield advantages context LLM-ST. Through rigorous experimentation on English Chinese datasets, showcase exceptional performance establishing new benchmark field translation. Demo: https://speechtranslation.github.io/llm-st/.