Reliability
Reliability: consistency or dependability of the data
Internal consistency: Measures reliability across items. Tests whether all items are measuring the same thing. Measured using Cronbach’s alpha
Test-retest reliability: Measures reliability over time. Tests whether data are consistent over time. Measured using correlations ®
Interrater reliability: Measures reliability across raters. Tests how consistent data are between different raters. Measured using either (a) kappa for categorical data or (b) interclass correlations (ICC) for numerical data.
Validity
Validity: the extent to which the test measure what it is intended to measure (as measured by correlation or relationship with another measure)
Convergent validity: correlation with another measure that is designed to test the same construct
Divergent validity: correlation (or lack thereof) with another measure designed to test a different construct
Predictive validity: the test’s ability to predict something (e.g., a future outcome) theoretically related to the construct being measured.
