Missing Data Imputation With Granular Semantics and AI-driven Pipeline for Bankruptcy Prediction

• DOI: 10.48550/arxiv.2404.00013
• Publication Date: 2024-01-01
• Authors: Chakraborty, Deba...; Ranjan, Ravi
• Keywords: FOS: Computer and information sciences; FOS: Economics and business; Computer Science - Machine Learning; Artificial Intelligence (cs.AI); Statistical Finance (q-fin.ST); Computer Science - Artificial Intelligence; Quantitative Finance - Statistical Finance; Applications (stat.AP); Statistics - Applications; Machine Learning (cs.LG)

15 pages<br/>This work focuses on designing a pipeline for the prediction of bankruptcy. The presence of missing values, high dimensional data, and highly class-imbalance databases are the major challenges in the said task. A new method for missing data imputation with granular semantics has been introduced here. The merits of granular computing have been explored here to define this method. The missing values have been predicted using the feature semantics and reliable observations in a low-dimensional space, in the granular space. The granules are formed around every missing entry, considering a few of the highly correlated features and most reliable closest observations to preserve the relevance and reliability, the context, of the database against the missing entries. An intergranular prediction is then carried out for the imputation within those contextual granules. That is, the contextual granules enable a small relevant fraction of the huge database to be used for imputation and overcome the need to access the entire database repetitively for each missing value. This method is then implemented and tested for the prediction of bankruptcy with the Polish Bankruptcy dataset. It provides an efficient solution for big and high-dimensional datasets even with large imputation rates. Then an AI-driven pipeline for bankruptcy prediction has been designed using the proposed granular semantic-based data filling method followed by the solutions to the issues like high dimensional dataset and high class-imbalance in the dataset. The rest of the pipeline consists of feature selection with the random forest for reducing dimensionality, data balancing with SMOTE, and prediction with six different popular classifiers including deep NN. All methods defined here have been experimentally verified with suitable comparative studies and proven to be effective on all the data sets captured over the five years.<br/>