The hidden Daubert factor: How judges use error rates in assessing scientific evidence

In Daubert v. Merrell Dow Pharmaceuticals, the United States Supreme Court provided a framework under which trial judges must assess the evidentiary reliability of expert scientific evidence. One factor of the Daubert test, the "known or potential rate of error" of the expert's method, has received considerably less scholarly attention than the other factors, and past empirical study indicates that judges have a difficult time understanding the factor and use it less frequently in their analyses as compared to other factors. In this Article, we examine one possible interpretation of the "known or potential rate of error" standard that would treat the factor more broadly: considering direct assessments of a method's validity as assessments of the method's potential rate of error, even when numerical error rates are not mentioned. To assess the extent to which judges use the error rate factor in this "implicit" sense, we examined 208 federal district court cases, coding for the number of words judges spent analyzing the Daubert factors and other evidentiary considerations. We found that judges faced with a Daubert challenge often undertake a detailed analysis of the quality of the methodology used by the expert rather than simply relying on proxies for the quality of the method such as peer review and general acceptance. Analysis of a method's potential rate of error was significantly more common and lengthy than analysis using any of the other Daubert factors. This implicit error rate analysis also predicted the final admissibility ruling of the evidence and varied across expert disciplines. Our data support the notion that judges put considerable effort into directly assessing the validity of the scientific evidence before them when responding to a Daubert challenge. That is, they engage substantially in central processing in making methodological evaluations rather than merely relying on the peripheral cues of peer review and general acceptance. This finding lays the groundwork for future assessments of the obstacles judges face in these demanding evaluations.