Reliability

• What is reliability a measure of in a study?

  Consistency of a study
• What does reliability include in a study?

  Consistency of procedures and measuring tools
• What are potential issues of reliability in a study?

  Procedures, measuring tool results, researcher
• What does it mean if a procedure lacks reliability?

  Something has not been done consistently
• How can you solve issues of reliability in procedures?

  Standardise procedures
• What happens if a measuring tool is unreliable?

  It does not consistently measure the same
• How can you solve issues with an unreliable measuring tool?

  Change the measuring tool
• What inconsistency might researchers face during observations?

  Different observations recorded by researchers
• How can researchers improve reliability in their observations?

  Use the same coding sheet
• What are the main issues of reliability in research?

  1. Procedure: Inconsistencies in instructions or tasks
  2. Measuring Tool: Inconsistent results over time
  3. Researchers: Inconsistent observations and recordings
• What is the test-retest method used for?

  Checking reliability of measuring tools
• How is the test-retest method conducted?

  Same test to same people on different occasions
• When does a measuring tool show reliability in the test-retest method?

  Scores are similar on both occasions
• What should be done if a test-retest shows poor reliability?

  Change the questionnaire
• What does inter-rater reliability check?

  Consistency among observers assessing behavior
• How is inter-rater reliability assessed?

  Observers use the same coding sheet independently
• When does inter-rater reliability show reliability?

  Scores of observers correlate well
• What should be done if inter-rater reliability shows poor reliability?

  Check behavioral categories and train observers
• What is the acceptable correlation for reliability?

  R=+0.8
• How can reliability be improved in different research methods?

  • Experiment: Control procedures in lab studies
  • Correlations: Check measuring tools for reliability
  • Observations: Use structured observations and multiple observers
  • Questionnaire: Use test-retest and closed questions
  • Interview: Use the same interviewer and structured format
• What is a benefit of lab-based experiments regarding reliability?

  Good control over procedures
• What should be done with questionnaires that are unreliable?

  Adapt them for clarity
• How can interviews improve reliability?

  Use the same interviewer for all participants
• What is a structured interview?

  An interview with a set format for questions