ROC-based model estimation for forecasting large changes in demand

Forecasting for large changes in demand should benefit from an estimation that differs from that used for estimating mean behaviors. We develop a multivariate forecast model whose continuous forecasts are used as test statistics in decision rules to make binary (yes/no) forecasts for large changes in time series data. The model is fit based upon a penalty function that maximizes the partial area under the ROC curve (PAUC) along a relevant range of false positive rates, and can be used by managers who wish to take action on the small percentage of products whose demand is likely to change the most in the next time period. We apply the model to a crime dataset and compare the results to OLS, as a basis for comparisons, as well as to models that have shown themselves to be promising for large-change demand forecasting, including logistic regression, quantile regression, synthetic data from a Bayesian model, and a power loss model. Using the PAUC metric, our proposed forecasting model's out-of-sample performance shows statistical significance, a 35% improvement over OLS, and at least a 20% improvement over other competing methods. We suggest that managers with large numbers of time series (e.g., for product demand) should use our method to forecast large changes preemptively, in conjunction with magnitude-based methods for forecasting the expected demand.