Reliability and Validity

• Reliability is a measure of whether something stays the same(is it consisten)
• Validity refers to whether a measure actually measures what it claims to be measuring.
• External reliability-The degree to which test gets the same or similar results on different occasions,with same participants.
• Internal reliability-internal consistency of a measure is the test consistent within itself.
• Assessing reliability

  Split-half method-randomly divide the test into 2 halves and compare the same p’s results.

  Inter Rater-2 observers on the same p’s and correlate results.

  Test-retest-carry out the same test on the same p’s twice.
• Improving reliability

  Tighten up behaviour categories in observations

  Train observers or interviewers in the use of the checklist or coding system to increase consistency

  Select test items that correlate most closely to improve the consistency of the test

  Use more than one observer/tester for each p and average the data to reduce observer bias

  Check questions/standardised instructions to ensure clarity & remove ambiguity
• Internal validity-The degree to which we are measuring what we intend to measure,whether results obtained are solely affected by changes in the variable being manipulated.
• External validity-whether data can be generalised to other situations outside the research environment they were originally gathered in.
• Types of external validity

  Ecological validity-examining whether conditions of an experiment mirror the natural environment being investigated.

  Temporal validity-validity of findings in relation to the progression of time.

  Population validity-how well the sample used can be extrapolated to the population as a whole.
• Assessing validity

  Face – degree to which research appears to be measuring what we want

  Concurrent – degree to which the study shows similar results to a previously validated measure

  Predictive – the degree to which it predicts future performance

  Ecological – the degree to which findings can be generalised to other situations

  Mundane realism – the degree to which the setting/procedures reflect real life

  Content – are the test items representative of that which is being measured

  Population – the degree to which the findings can be generalised from the sample to the target population
• Improving validity

  Single-blind and double/blind studies to control experimenter/observer bias

  Stratified/quota sampling to improve generalisability to target population

  Use peer review to determine the accuracy of the test in measuring what it is required to measure

  Use more than one observer/tester for each p and average the data to reduce observer bias

  Conduct the research in a variety of settings with variety of p’s