Metrics to evaluate R,D&E

Metrics affect research decisions, research efforts, and the researchers themselves. From a review of the literature, interviews at ten research-intensive organizations, and formal mathematical analyses, the authors conclude that the best metrics depend upon the goals of the R,D&E activity as they vary from applied projects to competency-building programs to basic research explorations. For applied projects, market outcome metrics (sales, customer satisfaction, margins, profit) are relevant if they are adjusted via corporate subsidies to account for short-termism, risk aversion, scope, and options thinking. The magnitude of the subsidy should vary by project according to a well-defined formula. For R,D&E programs that match or create core technological competence, outcome metrics must be moderated with "effort" metrics. Too large a weight on market outcomes leads to false rejection of promising programs. The large weight encourages the selection of lesser-value programs that provide short-term, certain results concentrated in a few business units. This, in turn, leads a firm to use up its "research stock." Instead, to align R,D&E with the goals of the firm, the metric system should balance market outcome metrics with metrics that attempt to measure research effort more directly. Such metrics include many traditional indicators. For long-term research explorations, the right metrics encourage a breadth of ideas. For example, many firms seek to identify their "best people" by rewarding them for successful completion of research explorations. However, metrics implied by this practice lead directly to "not-invented-here" attitudes and result in research empires that are larger than necessary but lead to fewer total ideas. Alternatively, by using metrics that encourage "research tourism," the firm can take advantage of the potential for research spillovers and be more profitable.