TY - JOUR

T1 - Non-Bayesian Testing of a Stochastic Prediction

AU - Dekel, Eddie

AU - Feinberg, Yossi

N1 - Funding Information:
Acknowledgements. Previous versions of this paper were circulated under the title “A True Expert Knows Which Question Should be Asked” (Dekel and Feinberg, 2004). We would like to thank John Conlon, Ehud Kalai, Wojciech Olszewski, Gil Reilla, Alvaro Sandroni, Eilon Solan, Bob Wilson, two referees, and the editor for helpful comments. The first author gratefully acknowledges the support of the NSF under grant 0111830. The second author gratefully acknowledges the support of the Center for Electronic Business and Commerce.

PY - 2006/10

Y1 - 2006/10

N2 - We propose a method to test a prediction of the distribution of a stochastic process. In a non-Bayesian, non-parametric setting, a predicted distribution is tested using a realization of the stochastic process. A test associates a set of realizations for each predicted distribution, on which the prediction passes, so that if there are no type I errors, a prediction assigns probability 1 to its test set. Nevertheless, these test sets can be "small", in the sense that "most" distributions assign it probability 0, and hence there are "few" type II errors. It is also shown that there exists such a test that cannot be manipulated, in the sense that an uninformed predictor, who is pretending to know the true distribution, is guaranteed to fail on an uncountable number of realizations, no matter what randomized prediction he employs. The notion of a small set we use is category I, described in more detail in the paper.

AB - We propose a method to test a prediction of the distribution of a stochastic process. In a non-Bayesian, non-parametric setting, a predicted distribution is tested using a realization of the stochastic process. A test associates a set of realizations for each predicted distribution, on which the prediction passes, so that if there are no type I errors, a prediction assigns probability 1 to its test set. Nevertheless, these test sets can be "small", in the sense that "most" distributions assign it probability 0, and hence there are "few" type II errors. It is also shown that there exists such a test that cannot be manipulated, in the sense that an uninformed predictor, who is pretending to know the true distribution, is guaranteed to fail on an uncountable number of realizations, no matter what randomized prediction he employs. The notion of a small set we use is category I, described in more detail in the paper.

UR - http://www.scopus.com/inward/record.url?scp=33749041141&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33749041141&partnerID=8YFLogxK

U2 - 10.1111/j.1467-937X.2006.00401.x

DO - 10.1111/j.1467-937X.2006.00401.x

M3 - Article

AN - SCOPUS:33749041141

SN - 0034-6527

VL - 73

SP - 893

EP - 906

JO - Review of Economic Studies

JF - Review of Economic Studies

IS - 4

ER -