NVMM-Oriented Hierarchical Persistent Client Caching for Lustre

• DOI: 10.1145/3404190
• Publication Date: 2021-01-18T13:16:30Z
• Authors: Wen Cheng; Chunyan Li; Lingfang Zeng; Yingjin Qian; Xi Li; André Brinkmann
• Keywords: Lustre (file system); Object storage; File server; Metadata management

In high-performance computing (HPC), data and metadata are stored on special server nodes client applications access the servers’ through a network, which induces network latencies resource contention. These typically equipped with (slow) magnetic disks, while store temporary fast SSDs or even non-volatile main memory (NVMM). Therefore, full potential of parallel file systems can only be reached if side storage devices included into overall architecture. this article, we propose an NVMM-based hierarchical persistent cache for Lustre system (NVMM-LPCC short). NVMM-LPCC implements two caching modes: read write mode (RW-NVMM-LPCC short) (RO-NVMM-LPCC integrates Hierarchical Storage Management (HSM) solution layout lock mechanism to provide consistent services I/O running nodes, meanwhile maintaining global unified namespace entire system. The evaluation results presented in article show that increase average throughput by up 35.80 times 9.83 compared native system, providing excellent scalability.