RELIABILITY

• Key point:

  • RELIABILITY = CONSISTENCY
• Ways of assessing reliability- Test-retest:

  • Test the same person twice
  • Same test- done on 2 or more different occasions
  • Test / Questionnaire results should be the same / similar (+0.8 or more = reliable)
• Ways of assessing reliability- Inter-observer:

  • Researchers observe the same behaviour independently (avoid bias)
  • Compare their data & correlate
  • +0.8 or more would suggest reliability 
• Improving Reliability- Questionnaires:

  • Rewrite Questions:
  • Low reliability -> questions may need to be removed / rewritten
  • Open questions -> closed questions (remove ambiguity)
• Improving Reliability- Interviews:

  • Improve training
  • If possible- the same interviewer each time
  • If not- all interviewers must be trained- avoid leading / ambiguous questions
• Improving Reliability- Observations:

  • Operationalisation of behavioural categories
  • BCs- should be operationalised and measurable 
  • Categories shouldn’t overlap (all possible behaviours included to avoid inconsistent results from other observers)
• Improving Reliability- Experiments:

  • Standardised Procedures
  • Procedures must be the same every time
  • Can compare the performance of different ppts