reliability

• Reliability


  A measure of consistency, meaning a study produces similar results when it's replicated.
• Reliability in Psychology

  • Questionnaires + interviews - use set lists of closed or open questions to measure opinions, personality traits and recalled past behaviours.
  • Experimental conditions - use a range of measures when testing participants between conditions. Experiments include EEG, cognitive tests or entire physical environments.
  • Observations - use clearly defined lists of operationalised behavioural categories to classify observed behaviours.
• External reliability


  The extent to which a measure is consistent when repeated (e.g. the results of a study are consistent with an exact replication at a different time and/or with different participants).
• Internal reliability


  The extent to which different parts of a measure are consistent with itself (e.g. if a 100-question IQ test is divided into two 50-question tests, the results for each set of questions with the same participant would be similar).
• Assessing internal reliability using the split-half method

  Split-half method - assess measures that test one variable with multiple questions, for example, questionnaires or tests of IQ.

  1. Split the test into two parts.

  2. Get participants to complete both parts

  3. Check how strongly the results of each half of the test correlate with the other half

  4. A strong correlation indicates internal reliability.
• Assessing external reliability using test-retest


  Repeat the study using the same procedures/measuring devices at different times and test the correlation between the two versions.
• Assessing inter-observer reliability


  1. Researchers create a list of Behavioral categories

  2. Two researchers separately use the same list to observe and record behaviors

  3. Compare their results to see if they're closely matched
• Improving reliability

  • For observations: Train observers, operationalize behavioral categories, pilot study to identify poor defined behavioural categories.
  • For interviews: Use structured format, carefully phrase questions, follow a script.
  • For questionnaires: Use closed questions, use established tests.
  • For experiments: Use clear standardized procedure, control environmental conditions, use established tests.
• Comparing with a test of correlation


  The level of correlation is assessed using a test of correlation such as Pearson's R or Spearman's Rho. A correlation of 0.8 or higher is usually accepted as a strong correlation.