TY - JOUR
T1 - Assessing the performance of direct marketing scoring models
AU - Malthouse, Edward C.
PY - 2001
Y1 - 2001
N2 - Direct marketers commonly assess their scoring models with a single-split gains chart method: They split the available data into "training" and "test" sets, estimate their models on the training set, apply them to the test set, and generate gains charts. They use the results to compare models (which model should be used), assess overfitting, and estimate how well the mailing will do. It is well known that the results from this approach are highly dependent on the particular split of the data used, due to sampling variation across splits. This paper examines the single-split method. Does the sampling variation across splits affect one's ability to distinguish between superior and inferior models? How can one estimate the overall performance of a mailing accurately? I consider two ways of reducing the variation across splits: Winsorization and stratified sampling. The paper gives an empirical study of these questions and variance-reduction methods using the DMEF data sets.
AB - Direct marketers commonly assess their scoring models with a single-split gains chart method: They split the available data into "training" and "test" sets, estimate their models on the training set, apply them to the test set, and generate gains charts. They use the results to compare models (which model should be used), assess overfitting, and estimate how well the mailing will do. It is well known that the results from this approach are highly dependent on the particular split of the data used, due to sampling variation across splits. This paper examines the single-split method. Does the sampling variation across splits affect one's ability to distinguish between superior and inferior models? How can one estimate the overall performance of a mailing accurately? I consider two ways of reducing the variation across splits: Winsorization and stratified sampling. The paper gives an empirical study of these questions and variance-reduction methods using the DMEF data sets.
UR - http://www.scopus.com/inward/record.url?scp=0012904873&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0012904873&partnerID=8YFLogxK
U2 - 10.1002/1520-6653(200124)15:1<49::AID-DIR1003>3.0.CO;2-F
DO - 10.1002/1520-6653(200124)15:1<49::AID-DIR1003>3.0.CO;2-F
M3 - Article
AN - SCOPUS:0012904873
SN - 1094-9968
VL - 15
SP - 49
EP - 62
JO - Journal of Interactive Marketing
JF - Journal of Interactive Marketing
IS - 1
ER -