Reliability

• What is reliability?

  The consistency of measurements - we would expect any measurement to produce the same data every time it is used
• What are two ways of assessing reliability?

  1. Test-retest

  2. Inter-observer reliability, inter-interviewer (for interviews), inter-rater (for content analysis)
• What is test-retest reliability?

  The same assessment given to the same participants on 2 occasions to see if the same results are obtained - these results are correlated and if they have a correlation coefficient of .80 or more = high test-retest reliability
• What is inter-observer reliability?

  The extent to which there is agreement between 2 or more observers involved in observations of a behaviour - if the scores have a correlation coefficient of .80 or more = good inter-observer reliability
• How is correlation carried out to assess the reliability of for example a questionnaire?

  - Each participants score from the first questionnaire should be correlated with their results from second questionnaire

  - Shown on a scattergraph, scores from the first test plotted on one axis and scores from second plotted on other

  - Strength of correlation assessed using either Spearman's rho or Pearson's r test
• How can researchers improve the reliability of an observation?

  Ensure the behavioural categories are operationalised clearly - no overlapping and are measurable
• How can researchers improve the reliability of a questionnaire?

  Ensure questions are not too complex - replace some of the open, ambiguous questions with closed, fixed choice alternatives
• How can researchers improve the reliability of an interview?

  If possible use the same interviewer each time - if not then ensure interviewers are properly trained and use more structured interviews.
• How can researchers improve the reliability of experiments?

  Ensure procedures are standardised