Measure Quality

• What needs to be considered in terms of measures?

  Precision and Accuracy

  Reliability and Validity
• Precision and Accuracy


  Precision= exactness (consistency) E.g. its 8:22 and 5 secs

  Accuracy= correctness (truthfulness) E.g. what is the actual time right now?
• Reliability and Validity


  Reliability= Precision (consistency) the extent to which our measure would provide the same results under the same conditions

  Validity= Accuracy (truthfulness) the extent to which it is measuring the construct we are interested in
• Forms of reliability


  Test-retest

  Inter-rater

  Parallel forms

  Internal consistency
• Test-retest reliability


  Measures fluctuations from one time to another

  If we administered our measure to the same participants on separate occasions, would we obtain the same results

  Important for constructs which we expect to be stable

  Beware of order effects
• Inter-rater reliability


  Measures fluctuations between observers

  If 2 different raters/observers measured the variable of interest, would the obtain the same results
• Parallel forms reliability


  If we administer different versions of our measure to the same participants, would we obtain the same results

  Different versions can be useful to help eliminate memory effects as the questions are different

  Beware of order effects and fatigue effects
• Internal consistency 


  Determines whether all items are measuring the same construct

  Can be assessed in a number of ways one e.g. split-half reliability: questionnaire items split into 2 groups and the halves are administered to Ps on separate occasions

  Beware of order effects
• Forms of Validity

  Content validity

  Face validity

  Criterion validity: Concurrent/Predictive

  Construct Validity: Convergent/Discriminant
• Content validity

  Does our test measure the construct fully?
• Face validity


  Does it look like a good test? Do the questions reflect what we're asking?

  Don't always want as it can lead to demand characteristics
• Criterion Validity 


  Does the measure give results which are in agreement with other measures of the same thing?

  Concurrent: Comparison of new test with established test

  Predictive: Does the test predict outcome on another variable
• Construct validity 


  Is the construct were trying to measure valid (does it exist)?

  The validity of a construct is supported by cumulative research evidence collected over time

  Convergent: correlates with tests of the same and related constructs

  Discriminant: doesn't correlate with test of different or unrelated constructs