Comparative testing of experts

Nabil I. Al-Najjar, Jonathan Weinstein

Research output: Contribution to journalArticlepeer-review

24 Scopus citations


We show that a simple "reputation-style" test can always identify which of two experts is informed about the true distribution. The test presumes no prior knowledge of the true distribution, achieves any desired degree of precision in some fixed finite time, and does not use "counterfactual" predictions. Our analysis capitalizes on a result of Fudenberg and Levine (1992) on the rate of convergence of supermartingales. We use our setup to shed some light on the apparent paradox that a strategically motivated expert can ignorantly pass any test. We point out that this paradox arises because in the single-expert setting, any mixed strategy for Nature over distributions is reducible to a pure strategy. This eliminates any meaningful sense in which Nature can randomize. Comparative testing reverses the impossibility result because the presence of an expert who knows the realized distribution eliminates the reducibility of Nature's compound lotteries. ©

Original languageEnglish (US)
Pages (from-to)541-559
Number of pages19
Issue number3
StatePublished - May 2008


  • Probability
  • Reputation
  • Testing

ASJC Scopus subject areas

  • Economics and Econometrics


Dive into the research topics of 'Comparative testing of experts'. Together they form a unique fingerprint.

Cite this