reliability

• Reliability

  • A measurement of consistency from an investigation or measuring device
  •  If a particular measurement is repeated and the same result is obtained then that measurement is described as being reliable
• Ways of assessing reliability

  • Test-retest
  • Inter-observer
  • Reliability is measured using correlation
  • In test-retest and inter-observer reliability, the two sets of scores are correlated
  • The correlation coefficient should exceed +.80 for reliability
• test-retest:

  • Test the same person twice
  • The same test or questionnaire is given to the same person on 2 or more different occasions
  • If the test or questionnaire is reliable the results should be the same/very similar (+.80 correlation) each time it is administered
  • Things to be careful of:
  • Test taken again too close to original time = pps may remember answers
  • Test taken too far away from original time = pps opinions may change
• inter-observer

  • Compares observations from different observers
  • observers should watch the same event/sequence of events but record their data independently
  • A pilot study should be conducted to ensure that all observes are applying behavioural categories in the same way
• improving reliability in:

  • Questionnaires = rewrite questions (eg if previous questions performed poorly on test-retest, may get rid of some open questions etc)
  • Interviews = improved training (leads to increased consistency)
  • Observations = operationalisation of behavioural categories (categories should be measurable and not overlap)
  • Experiments = standardised procedures (leads to increased consistency)