Reliability and Validity

• difference between reliability and validity

  reliability

  - measure of consistency

  - helps to promote confidence in researcher's psychological tests, observations + questionnaires

  validity

  - refers to accuracy
• External reliability


  between tests

  do we get the same results after time using the same test or doing the same investigation
• Internal reliability


  within one test

  are results consistent within a test?

  if we split the test in half, do both halves show the same thing?
• how can we improve reliability?

  interviews

  use same interviewer(s) must be trained so question aren't leading or ambiguous

  
  

  observations

  ensure behaviours categories are operationalised -> behaviour can overlap due to interpretation.

  observer may need training to ensure its more consistent

  
  

  experiments

  standardised the procedure, we can see if the results are reliable with different ppts -> if consistent - high reliability

  
  

  pilot studies

  standardising measurement
• Test re-test


  administer the same test or questionnaire to the same group/person on different occasions

  different occasions need to be sufficient time between both

  if 2 sets of scores are correlated, if positive then it's significant -> suggests reliability is good
• Split-half


  data is split in half

  results are compared

  correlate the 2 scores

  if correlation is above +0.8, result are reliable
• Inter-rater


  observational research that subjected to bias from observer's -> reduces reliability

  extent to which an agreement between observers involved in observation

  total number of agreements/total number of observations +0.8 -> high inter-observer reliability

  similar methods = content analysis (inter-rater) + interviews
• External validity


  refers to factors outside the experiment such as generalisability of experimental results

  examples

  population - generalising results to population

  ecological - generalising results to other settings

  temporal - whether findings from a particular study hold true over time
• how might we improve external validity?

  representative sample

  random, representative sample raises ethical/conflict

  can't force ppts to take part so will always have a biased sample

  ecological validity can be increased by making the task 'realistic'

  often a balance to be struck between external

  + internal validity between realism + control
• Internal validity


  effects observed in an experiment are due to manipulation of IV and not other factors
• what factors may affect internal validity?

  demand characteristics

  screw you effect

  evaluation apprehension

  confounding variables - ppt variables

  experimenter bias

  investigator effect

  operationalisation of variables
• how might internal validity be improved?

  confounding variables

  - need to be removed to improve internal validity -> purpose of control

  - ppt bias

  - single-blind -> this is where the ppt doesn't know what condition they're in or the aim of study

  
  

  experimenter bias

  double blind -> where the ppts + the experimenter don't know what condition they're in or the aim of the study
• Concurrent validity


  test or scale is demonstrated when result is checked against and are close to or match those obtained on another recognised test
• Content validity


  getting an expert in that particular field to review you measurements to see if they actually measure what you intent to measure
• Face validity


  taking the face and measure what is supposed to be measured by eyeballing

  reviewed by an expert