Reliability

• What does reliability refer to in assessments?

  Consistency of a measuring device or assessment
• What is test-retest reliability?

  Assessing the same person on two occasions
• Why is sufficient time needed between test and retest?

  To prevent recall of previous answers
• What are the two ways of measuring reliability?

  • Test-retest
  • Inter-observer Reliability
• What can happen if retesting occurs after a long period?

  Participants' attitudes or abilities may change
• What does inter-observer reliability measure?

  Agreement between two or more observers
• How can inter-observer reliability be established?

  1. Pilot study to check observer agreement
  2. Comparison of results at study's end
• How is reliability measured in test-retest and inter-observer reliability?

  By correlating the two sets of scores
• What is the accepted reliability correlation coefficient?

  0.8 or higher
• What are some methods to improve reliability in questionnaires?

  • Rewrite items for low test-retest reliability
• How can reliability be improved in interviews?

  • Use the same interviewer
  • Train all interviewers properly
• What steps can improve reliability in observations?

  • Operationalise all behavioural categories
  • Train observers for consistency
• How can experiments improve reliability?

  • Use standardised procedures consistently