Towards Automated Movie Trailer Generation

• DOI: 10.48550/arxiv.2404.03477
• Publication Date: 2024-06-16
• Authors: Argaw, Dawit Mureja; Soldan, Mattia; Pardo, Alejandro; Zhao, Chen; Heilbron, Fabian ...; Chung, Joon Son; Ghanem, Bernard
• Keywords: FOS: Computer and information sciences; Computer Vision and Pattern Recognition (cs.CV); Computer Science - Computer Vision and Pattern Recognition

Movie trailers are an essential tool for promoting films and attracting audiences. However, the process of creating trailers can be time-consuming and expensive. To streamline this process, we propose an automatic trailer generation framework that generates plausible trailers from a full movie by automating shot selection and composition. Our approach draws inspiration from machine translation techniques and models the movies and trailers as sequences of shots, thus formulating the trailer generation problem as a sequence-to-sequence task. We introduce Trailer Generation Transformer (TGT), a deep-learning framework utilizing an encoder-decoder architecture. TGT movie encoder is tasked with contextualizing each movie shot representation via self-attention, while the autoregressive trailer decoder predicts the feature representation of the next trailer shot, accounting for the relevance of shots' temporal order in trailers. Our TGT significantly outperforms previous methods on a comprehensive suite of metrics.<br/>Accepted to CVPR 2024<br/>