Saturday workshop: Mismeasurement in Program Evaluation

Mismeasurement in Program Evaluation

with Craig W. LeCroy, PhD from LeCroy & Milligan Associates

Progress in evidence-based practice has been substantially constrained by systematic mismeasurement. Although reliability and validity are widely treated as sufficient criteria for selecting instruments, many commonly used measures are poorly suited to detecting intervention effects. Drawing on examples from home visitation research, the workshop demonstrates how instruments originally designed to measure constructs or predict risk—such as parenting stress or child abuse potential—are routinely misapplied as outcome measures, resulting in null or mixed findings that may reflect measurement failure rather than program ineffectiveness. The provocative presentation critiques conventional research practices that privilege habituation and psychometric tradition over thoughtful alignment between intervention goals and measurement strategies. Central to the argument is the concept of sensitivity to change, or “validity for change,” which is often neglected despite being essential for outcome evaluation.