Accelerator-as-a-Service in Public Clouds: An Intra-Host Traffic Management View for Performance Isolation in the Wild

• DOI: 10.48550/arxiv.2407.10098
• Publication Date: 2024-07-14
• Authors: Jiechen Zhao; Ran Shu; Katie Lim; Zewen Fan; Thomas Anderson; Mingyu Gao; Natalie Enright J...
• Keywords: Isolation

I/O devices in public clouds have integrated increasing numbers of hardware accelerators, e.g., AWS Nitro, Azure FPGA and Nvidia BlueField. However, such specialized compute (1) is not explicitly accessible to cloud users with performance guarantee, (2) cannot be leveraged simultaneously by both providers users, unlike general-purpose (e.g., CPUs). Through ten observations, we present that the fundamental difficulty democratizing accelerators insufficient isolation support. The key obstacles enforcing accelerator are too many unknown traffic patterns possible contention sources datapath. In this work, instead scheduling complex on-the-fly augmenting support on each system component, propose model as network flows proactively re-shape avoid unpredictable contention. We discuss implications our findings design future management stacks device interfaces.