Reliability and validity

• Reliability:

  CONSISTENCY of results
• Inter-observer reliability:

  1. Researchers must have the SAME training and use the same behaviour categories
  2. Two psychologists conduct the observation/content analysis/ study independently
  3. They check for agreement and if they find similar results it is reliable
  4. This could be done by using a correlation, a strong positive correlation of +0.8 shows the observers gave similar results - so has reliability
• Inter-observer reliability simplified:

  1. Same training
  2. Independently study
  3. Agreement
  4. Strong positive correlation of +0.8
• Improving inter-observer reliability:

  Training: Give the observers better training so they can identify the behaviour

  Operationalisation: Make sure both observers are clear on what theyre looking for - could be checked using pilot study

  View of behaviour: Video the behaviour and let the observers view the same behaviour
• Test-retest reliability:

  1. A test would be given to a participant on two different occasions and the scores are compared
  2. If they find similar results it is reliable
  3. This could be done by using a strong positive correlation of +0.8 or above
• Validity: extent to which a test measures what it intends to measure
• Internal validity:

  • findings are due to the IV and nothing else
  • We can establish a true cause and effect relationship

  Improving:

  • Extraneous variables must be minimised - controlled environment EVs are easier to control
• External validity: Ecological validity

  • Results can be generalised to the real world

  Improving:

  • conduct in more natural environments
  • Use tasks that represent everyday life
  • Use materials that represent everyday life
• External validity: population validity

  • results can be generalised to the rest of the target population

  Improving:

  • Conduct studies on large sample sizes
  • Use random/stratified sampling
  • Conduct research cross-culturally and across different ages and genders
• External validity: Temporal validity

  • results can be generalised to the modern day

  Improving:

  • do the test again in the modern day to see if results are still accurate
• Assessing validity: Concurrent validity:

  • Involves assessing how closely the scores on a test match a different measurement
  • High concurrent validity is when there is a close agreement between the data produced by the new test compared to the established test - indicated if both tests exceeds +0.8