Privacy-Preserving Sharing of Data Analytics Runtime Metrics for Performance Modeling

• DOI: 10.1145/3629527.3652276
• Publication Date: 2024-05-07T16:19:06Z
• Authors: Jonathan Will; Dominik Scheinert; Seraphin Zunzer; Jan Bode; Cedric Kring; Lauritz Thamsen
• Keywords: Data Sharing; Data Analysis

Performance modeling for large-scale data analytics workloads can improve the efficiency of cluster resource allocations and job scheduling. However, performance these is influenced by numerous factors, such as inputs assigned resources. As a result, models require significant amounts training data. This be obtained exchanging runtime metrics between collaborating organizations. Yet, not all organizations may inclined to publicly disclose metadata.