Generative Table Pre-training Empowers Models for Tabular Prediction

• DOI: 10.18653/v1/2023.emnlp-main.917
• Publication Date: 2023-12-10T21:58:19Z
• Authors: Tianping Zhang; Shaowen Wang; Shuicheng Yan; Li Jian; Qian Liu
• Keywords: Table (database); Training set; Lookup table

Recently, the topic of table pre-training has attracted considerable research interest. However, how to employ boost performance tabular prediction remains an open challenge. In this paper, we propose TapTap, first attempt that leverages empower models for prediction. After on a large corpus real-world data, TapTap can generate high-quality synthetic tables support various applications including privacy protection, low resource regime, missing value imputation, and imbalanced classification. Extensive experiments 12 datasets demonstrate outperforms total 16 baselines in different scenarios. Meanwhile, it be easily combined with backbone models, LightGBM, Multilayer Perceptron (MLP) Transformer. Moreover, aid pre-training, trained using data generated by even compete original dataset half experimental datasets, marking milestone development generation. The code are available at https://github.com/ZhangTP1996/TapTap.