reliability + validity

• reliability

  refers to how consistent a measuring device is
• what are the ways of assessing reliability?

  • test-retest
  • inter-observer
• test-retest

  • method of assessing reliability by assessing the same person on two separate occasions.
  • shows to what extent the test produces the same answers
• inter-observer

  • the extent to which there is agreement between two or more observers involved in observations of a behaviour
• measuring reliability

  • uses a correlational analysis
  • correlation coefficient should exceed +.80 for reliability
• improving reliability in questionnaires

  • uses test-retest method
  • some items may have to be deselected or rewritten if the questionnaire produces low test-retest reliability
  • e.g. if they are complex or ambiguous
  • replace open questions with fixed choice alternatives
• improving reliability in interviews

  • use the same interviewer every time
  • interviewers must be properly trained
  • can't ask too ambiguous or leading questions
  • structured interviews
• improving reliability in observations

  • make sure behavioural categories have been properly operationalised
  • categories shouldn't overlap
  • observers may need training in using behavioural categories.
• improving reliability in experiments

  • procedures need to be standardised.
• validity

  • the extent to which an observed effect is genuine
  • does it measure what it was supposed to measure and can it be generalised?
• internal validity

  • refers to whether the effects observed in an experiment are due to the manipulation of the independent variable and not another factor.
  • major threat to internal validity is demand characteristics
• external validity 


  • relates to factors outside of the investigation.
• ecological validity

  • form of external validity
  • concerns generalising the findings from a study to other settings.
• temporal validity

  • form of external validity
  • the extent to which findings from a research study can be generalised to other times - does it hold true over time?
• what are the ways of assessing validity?

  • face validity
  • concurrent validity
• face validity

  • whether a measures appears to measure what it is supposed to measure.
  • e.g. does a test of anxiety look like it measures anxiety?
• concurrent validity

  • the extent to which a psychological measure relates to an existing and well-established, similar measure
• improving validity in experiments

  • using a control group to assess whether the changes in the dependent variable were due to the manipulation of the independent variable.
  • use standardised procedures to minimise impact of participant reactivity + investigator effects.
• improving validity in questionnaires

  • incorporate a lie scale within the questions to assess consistency of responses and to control for effects of social desirability bias.
  • enhance validity by assuring respondents that their data will be anonymous
• improving validity of observations

  • researcher remains undetected - covert
  • make sure behavioural categories are specific, don't overlap and clear
• improving validity in qualitative research

  • interpretive validity - extent to which researcher's interpretation of events matches their participants
  • include direct quotes from participants in their reports
  • triangulation - use a number of different sources.