An Object Storage for Distributed Acoustic Sensing

• DOI: 10.1785/0220230172
• Publication Date: 2023-10-20T11:51:47Z
• Authors: Yiyu Ni; Marine A. Denolle; Rob Fatland; Naomi Alterman; Bradley P. Lipovsky; Friedrich Knuth
• Keywords: Object storage; Schema (genetic algorithms); Distributed data store; Benchmark (surveying)

Abstract Large-scale processing and dissemination of distributed acoustic sensing (DAS) data are among the greatest computational challenges opportunities seismological research today. Current formats computing infrastructure not well-adapted or user-friendly for large-scale processing. We propose an innovative, cloud-native solution DAS seismology using MinIO open-source object storage framework. develop schema cloud-optimized formats—Zarr TileDB, which we deploy on a local service compatible with Amazon Web Services (AWS) system. benchmark reading writing performance various canonical use cases in seismology. test our framework server AWS. find much-improved compute time memory throughout when TileDB Zarr compared to conventional HDF5 format. demonstrate platform heavy case seismology: ambient noise data. process one month data, pairing all 2089 channels within 24 hr AWS Batch autoscaling.