Reliability across all methods of investigation

• Reliability= consistency
• If a particular measurement is repeated and the same result is obtained then that measurement is described as being reliable
• What are the three ways to assess reliability?

  1. Test-retest
  2. Inter-observer
  3. Correlations
• Test-retest: test the same person twice

  The same test or questionnaire is given to the same person (or people) on two or more different occasions

  If the test or questionnaire is reliable the results should be the same (or very similar) each time it is administered
• Inter-observer: compares observations from different observers

  In an observation, two or more observers compare their data by conducting a pilot study- a small-scale trial run of the observation to check that observers are applying behavioural categories in the same way

  Observers should watch the same event, or sequence of events, but record their data independently
• Correlation:

  In test-retest and inter-observer reliability, the two sets of scores are correlated. The correlation coefficient should exceed +.8 for reliability
• What are the ways to improve reliability in different methods?

  1. Questionnaires- rewrite questions
  2. Interviews- improved training
  3. Experiments- standardised procedures
  4. Observations- operationalisation of behavioural categories
• Improve questionnaires: rewrite questions:

  A questionnaire that produces low test-retest reliability may need some items to be deselected or rewritten

  The researcher may replace some open questions (which can be misinterpreted) with closed, fixed choice alternatives which may be less ambiguous
• Improve interviews: improved training:

  The best way of ensuring reliability in an interview is to use the same interviewer each time

  If this is not possible, all interviewers must be trained (e.g. so they avoid questions that are leading or ambiguous)
• Improve experiments: standardised procedures:

  Lab experiments are often described as being reliable because of the strict control over many aspects of the procedure, such as the instructions that the participants receive and the conditions within which they are tested
• Improve observations: operationalisation of behavioural categories:

  Behavioural categories should be measurable (e.g. 'pushing' is less open to interpretation than 'aggression')

  Categories should not overlap (e.g. 'hugging' and 'cuddling') and all possible behaviours should be included

  If categories are overlapping or absent, different observers have to use their own judgement in deciding what to record and where, and may end up with inconsistent records