Reliability

• Definition

  How consistent the findings from an investigation or measuring device are. A measuring device is said to be reliable if it produces consistent results every time it is used.
• Ways of assessing reliability

  • Test-retest method
  • Inter-observer reliability
• Test-retest method - a method of assessing the reliability of a questionnaire, psychological test or interview.
• How can a test-retest method be used?

  This assesses the same person on two separate occasions.

  If the test produces the same or very similar answers it is consistent or reliable
• Time between test-retest

  There has to be enough time between test and retest to allow the participant enough time not to remember the test but not too long so that their attitudes, opinions may have changed
• Results of test-retest method

  The two sets of scores are correlated and if the correlation is significant and positive (at leat +0.8) then it is reliable
• Inter observer reliability - a method of assessing the reliability of an observation
• How can inter observer reliability be used?

  It involves the same event or sequence of events being observed by two observers at the same time but independently of each other.
• Results of inter-observer reliability

  The data from the observations is then correlated and if the correlation is significant and positive (at least +0.8) it is assumed that the observation and behavioural categories were reliable.
• The same process of inter observer reliability can be used for content analysis (inter-rater reliability) and interviews (inter-interviewer reliability)
• Ways of improving reliability - questionnaires

  Questions may need to be deflected or rewritten

  Removing complex or ambiguous questions

  Such as chaining open questions to closed
• Ways of improving reliability - interview

  Better training for interviewers

  Using the same interviewer

  Using structured rather than unstructured interviews
• Ways of improving reliability - experiments

  High levels of control help to ensure reliability

  Lab experiments are better than field or nature

  Ensures replication
• Ways of improving reliability - observations

  Make sure behaviour categories are properly operationalised and measurable

  Better training for observers