The theory is predictive, but is it complete? an application to human perception of randomness

When we test theories, it is common to focus on what one might call predictiveness: how well do the theory's predictions match what we see in data? Evidence that a theory is predictive, however, provides little guidance towards whether there may exist alternative theories that are more predictive, and how much more predictive they might be. These questions point toward a second issue, distinct from predictiveness, which we call completeness: how close is the performance of a given theory to the best performance that is achievable in the domain? Completeness is an important construct because it lets us ask how much room there is for improving the predictive performance of existing theories in any given domain. We would expect the best possible prediction performance to di.er considerably across domains-for example, an accuracy of 55% is a stunning success for predicting stock movements based on past returns, but extremely weak for predicting movements of a planet based on physical measurements. .is contrast rejects that variation in stock movements conditioned on the features we know is large, while planetary motions are well predicted by known features. To understand how much we can improve on the predictive performance of existing theories, we need to separate prediction error due to intrinsic noise (emerging from limitations of the feature set) from prediction error that reveals opportunities for a be.er model.