Minimization of high computational cost in data preprocessing and modeling using MPI4Py

• DOI: 10.1016/j.mlwa.2023.100483
• Publication Date: 2023-07-17T09:33:33Z
• Authors: E. Oluwasakin; T. Torku; S. Tingting; A. Yinusa; S. Hamdan; S. Poudel; N. Hasan; J. Vargas; K. Poudel
• Keywords: Overfitting; Data pre-processing

Data preprocessing is a fundamental stage in deep learning modeling and serves as the cornerstone of reliable data analytics. These models require significant amounts training to be effective, with small datasets often resulting overfitting poor performance on large datasets. One solution this problem parallelization modeling, which allows model fit more effectively, leading higher accuracy sets overall. In research, we developed novel approach that effectively deployed tools such MPI MPI4Py from parallel computing handle processes. As case study, technique applied COVID-19 state Tennessee, USA. Finally, effectiveness our demonstrated by comparing it existing methods without concepts like MPI4Py. Our results demonstrate promising outcome for deployment minimize high computational cost.