Reliability

• what is reliability?

  Consistency. how much we can depend on a particular measurement. if we repeat the same study/test/measurement will it get the same result
• what does it mean if you make a particular measurement twice and the result is the same?

  the measurement is reliable
• what if there's a change to the measurement over time?

  we would attribute that change to the object rather than the 'ruler'
• taking data on successive occasions

  the measurement should produce the same data every time to be reliable
• what is the correlation results must be to be reliable?

  0.8+
• what are the ways of assessing reliability?

  Test-retest

  Inter-observer
• basics for test-retest

  Test the SAME person TWICE
• What is test-retest?

  -administering the same test/ questionnaire to the same person on different occasions

  -if the test is reliable they should have the same/ very similar results

  -there must be sufficient time between so they can't recall answers

  -2 sets of scores will be correlated to make sure they're similar

  -if correlation turns out significant (0.8+) then the measurement of the instrument is assumed reliable
• basics for the inter-observer

  Comparing DIFFERENT observations from different OBSERVERS
• what is inter-observer?

  -observational research

  -observers have to have the same outcome and opinions

  -agree on how they assess

  -correlate and 0.8+ = reliable

  (content analysis= inter-rater, Interview inter-interview)
• what are the ways of improving reliability?

  Questionnaires

  Interviews

  Observations

  Experiments
• what does QIOE stand for?

  questionnaires, interviews, observations, experiments- ways of improving reliability
• Basics of questionnaires- reliability

  REWRITE the question
• How do you improve reliability through questionnaires?

  a questionnaire that produces a low test-retest reliability may require some items to be deselected or rewritten

  eg/ replace some open questions with closed as they may be ambiguous
• Basics of interviews- reliability

  Improve the TRAINING
• How do you improve reliability through interviews?

  All interviewers must be trained

  eg/ 1 might be asking questions too leading/ambiguous

  Easier in structured interviews as interviewer behaviour is more controlled by the fixed questions
• Basics of observations- reliability

  OPERATIONALISATION of behavioural categories
• How do you improve reliability through observations?

  Reliability can be improved by ensuring behavioural categories have been operationalised

  and they are measurable and self-evident

  if categories aren't properly operationalised different observers have to make their own judgements of what to record and end up with differing and inconsistent records
• Basics of experiments- reliability

  SANDARDISE procedures
• How do you improve reliability through experiments?

  in order to compare the performance of different participants the procedures must be the same every time= standardised procedures

  in an experiment its the procedures that are the focus of reliability