Validity and reliability

• Validity


  Data accuracy
• Internal validity


  What goes on inside the research (data accuracy)
• Pilot study


  Small scale study conducted in the way the real research would happen. Flaws can be pointed out, participants can be asked if they understood the instructions. Participants used in the pilot cannot be used in the real experiment.
• External validity


  Factors outside the investigation (if research is generalisable)
• Ecological validity


  Can findings be generalised to settings beyond the study
• Lab experiments can lack ecological validity as it’s not in a natural setting
• Mundane realism


  If the study was like ‘real everyday life’
• Temporal validity


  Relevant to the time period affecting the findings
• Face validity


  Subjective. Your opinion On if it’s valid.
• Content validity


  Subjective. Experts look over and evaluate whether the test is measuring what it should.
• Concurrent validity


  Objective. If results match results on a similar test.
• Construct validity


  Objective. If other studies match results on a similar test
• Predictive validity


  Objective. If predictions made have been shown to come to pass
• Improve validity- experiments

  • Control group- researcher can confirm a change in the DV was due to the IV
  • Standardised procedures- Minimise participant reactivity and investigator effects
  • Single/double blind procedures- reduce demand characteristics and investigator effects
• Improve validity- questionnaires

  • Lie scale- respondent are being consistent and not showing social desirability bias
  • Anonymity- participants don't need to lie
• Improve validity- observations

  • Covert, naturalistic observations
  • Precise behavioural categories- data collected is accurate
• Improve validity- qualitative methods

  • Higher ecological validity- reflects participants reality
  • Direct quotes- not misinterpreting information
• Realiable

  Measure of consistency

  If the measure can be repeated and the results are replicated it‘s described as reliable
• How to improve reliability


  Develop more consistent forms of measurement with clearly defined operational definitions and by improving inter-observer reliability
• Internal reliability


  The extent which something is consistent within itself
• External reliability


  Tests measures consistently over time
• Test retest (external)

  1. Person completes the test/questionnaire etc…
  2. Given the same test on a different occasion
  3. Some time is left in between
  4. 2 results are correlated and analysed
  5. Produces a correlation coefficient- 0.8 or higher
  6. 2 sets of scores don’t have to be the same to be reliable
• Problems with test retest


  Practice effect- better the 2nd time

  Fatigue effect- worse the 2nd time
• Split-half reliability/internal consistency (internal)


  Tests how reliable a test is within itself

  1. Compares participant scores on half the questions with the scores on the 2nd half
• Inter-observer reliability (external)

  • links to observations
  • If there’s more than 1 observer they must ensure they’re interpreting the same way
  • The results are correlated and analysed with a statistical test (0.8 or high)
  • Issue of subjectivity bias so there should be more than 1 observer
• Improving reliability- experiments

  Lab experiments are reliable as there‘s control of many variables
• Improving reliability- questionnaires

  If test retest is lower than 0.8 some questions can be changed or removed
• Improving reliability- Observations


  Make sure behavioural categories have been operationalised
• Improving reliability- qualitative methods


  Same trained interviewer and structured interviews to avoid leading questions
• Inter-rated reliability


  Reliability between you and another researcher (consistency)
• Intra-rater reliability


  Reliability between yourself