Optimal threshold for Pareto tail modelling in the presence of outliers

• DOI: 10.1016/j.physa.2018.06.007
• Publication Date: 2018-06-14T23:53:16Z
• Authors: Muhammad Aslam Mo...; Nurulkamal Masseran; Kamarulzaman Ibrahim
• Keywords: 0103 physical sciences; 1. No poverty; 01 natural sciences

Abstract The Pareto distribution is widely applied in many areas of studies such as economics and sciences. An important issues related to Pareto tail modelling is to determine the optimal threshold of the Pareto distribution. One of the methods used for determining the optimal threshold of Pareto distribution is by choosing the threshold that minimizes the goodness-of-fit statistics found based on empirical distribution function (EDF). This study involves determination of the shape parameter of the Pareto distribution using the maximum likelihood method and robust method based on the probability integral transform statistics. In addition, given the particular estimates of the shape parameter, comparison of the performance of several EDF statistics, namely, Kolmogorov–Smirnov, Kuiper, Anderson–Darling, Cramer–von Misses and Watson statistics in determining the optimal threshold in the presence of outliers is studied based on Monte Carlo simulation. Since the EDF statistics are found smallest for Kolmogorov–Smirnov or Kuiper statistics, these two EDF statistics outperformed other EDF statistics considered. The findings are illustrated using a sample of household income data of the Malaysian population. The optimal threshold found can be used to classify the high income earners in Malaysia since Pareto distribution is one of the most frequently used model to describe the upper tail of income distribution.