Reliability and Validity of Observations

• Controlled observations - High reliability, controlled environment, high control over extraneous variables
• Naturalistic observations - Low reliability, real life environment, low control over extraneous variables
• Inter-observer reliability can be used to assess reliability of observations
• How to conduct inter-observer reliability:

  1. The reliability of observations can be checked by using two observers
  2. The two observers would create and be trained on how to use the behaviour categories (AO2)
  3. Two observers would then conduct the observation separately - watch exactly the same behaviour (AO2) for the same amount of time but independently record/tally their observations
  4. The tallies from both observers should then be compared and correlated using an appropriate stats test
  5. A strong positive correlation of +0.8 shows high reliability
• Operationalising can be used to improve reliability of observations
• Operationalising: To be specific and clear when defining any behaviour categories (1), so they can be easily measured (1)
• Operationalising is important because if behaviour categories are vague then it would not be possible for two more observers to conduct the same observation to check for consistent results as they may not be looking for the exact same behaviours
• Operationalising increases reliability as if variables are operationalised, the other researchers can conduct the same observation in the same way to check for consistent results as they know exactly what they are looking for
• Face validity can be used to assess the validity of observations
• How to conduct face validity:

  1. The quickest most superficial way of assessing for validity
  2. An independent psychologist in the same field looks at the behaviour categories (AO2) to see if they look like they measure what they intend to measure (AO2) at first sight/face value
  3. If the researcher says 'yes' then the observation is valid
• Concurrent validity can also be used to assess validity of observations
• How to conduct concurrent validity:

  1. Compare the results of the new observation (AO2) with the results from another similar pre-existing observation which has already been established for its validity
  2. Correlate the two sets of behavioural recordings/results gained from an appropriate stats test, this should exceed +0.8
  3. If results gained from both observations are similar, then we can assume the test is valid
• To improve validity in observations:

  • Ensure behaviour categories are operationalised
  • Observers are trained in how to use the behaviour categories
  • Use covert observations as participants behaviour are more likely to be natural