Validity, Reliability, and Pilot Studies

• Validity:

  • Validity is a measure of the truth. It tests if the tool you are using measures what it intends to measure.
• Internal Validity:

  • How much do the findings of the dependant variable have to do with the independent variable and no other factors
• External Validity:

  • All about whether our results can be generalised beyond the study.
• Population Validity:

  • External validity – can they be generalised to wider population.
• Ecological Validity:

  • Can the results be generalised to the real world.
• Lack of mundane realism:

  • Participants are asked to perform something atypical.
• Temporal Validity:

  • The findings of a study or concepts of a particular theory hold true.
• Testing Validity:

  • Face validity:
  • When an independent expert looks at the measure being used and assesses if it will measure its aim.
  • Concurrent Validity:
  • Comparing the new procedure with a similar procedure that has been done before and where the validity has been established.
  • Predictive Validity:
  • The extent to which a score on a scale or test predicts future score on the same measure.
• Improving Validity:

  • Controlling more variables.
  • Improving measurement techniques, increasing randomisation.
  • Reducing sample bias.
  • Adding control groups.
• Reliability:

  • Reliability is a measure of consistency, if a particular measurement is repeated and the same result is obtained, then it is described as reliable.
  • If a test lacks reliability, it also lacks validity
• External Reliability:

  • This measures consistency from one occasion to another.
  • To improve external reliability, we can do TEST-RETEST, where participants do the same test on different occasions.
  • High correlation between the results means high external reliability.
• Internal Reliability:

  • This measures the extent to which a test is consistent within itself.
  • Can be assessed using SPLIT-HALF method.
  • If the two halves of the test produce similar results, it implied that the test has internal reliability.
• Improving Reliability: Inter-rater reliability:

  • The consistency of a researcher’s behaviour.
  • A researcher should carry out interviews in the same way more than once.
• Improving Reliability: Improving questionnaires:

  • Should be measured using test-retest and then compared with two data sets.
  • If this produces a low test-retest reliability then some items may need to be re-written.
• Improving Reliability: Improving Interviews:

  • Best way is to use the same interviewer each time.
  • Interviewer must be trained to avoid ambiguity.
• Improving Reliability: Improving experiments:

  • Easily repeated = more reliable.
• Improving Reliability: Improving observations:

  • These can be improved by operationalising behaviour categories – making sure they are measurable and specific.