Repeatability and Reproducibility: Are your measurements precise enough to analyze?

The rapidly exploding interest in predictive analytics has increased dramatically the number of people applying statistical techniques to their data. Too often, analysts take whatever data they can find, enter it into a statistical package, run a procedure or two, and go directly to the final results. They fail to realize that all data is subject to some amount of measurement error. Unless the analyst has a good handle on how large that error is, it is hard to judge the reliability and applicability of the final results.

Prior to the use of any statistical procedure, it is important to understand how repeatable and reproducible the individual data values are. By repeatable, we mean the potential difference between the results at hand and the results that might be obtained if we repeated our study under similar conditions. By reproducible, we mean the potential difference between the results that we obtained and the results that someone else might get if they tried to reproduce our study.

The measurement systems analysis and Gage Studies section of Statgraphics Centurion contains 8 procedures designed to help estimate Repeatability and Reproducibility (R&R). 5 of those procedures deal with variable data (measurements) while the other 3 deal with attribute data. In the variable studies, multiple appraisers make repeated measurements on identical samples (or close to identical samples if testing is destructive). In attribute studies, multiple appraisers repeatedly rate selected items. Statistics calculated from such studies, such as the Precision/Tolerance Ratio, the Percent of Total Variation, the Number of Distinct Categories, and Cohen’s Kappa Measure of Interrater Agreement  let the analyst know whether their measurement system is good enough to expect usable results from other analyses.

Note: if your data come from a destructive test that can’t be repeated, check out How to Perform a Repeatability and Reproducibility Study for Destructive Tests Using STATGRAPHICS at http://www.statgraphics.com/how-to-guides