validity and reliability

• validity


  when a test measures what it claims to measure

  measurements shouldn't be effected by any extraneous or confounding variables
• assessing validity


  face validity: when we look at a test to see if it measures what it claims to be measuring (at face value)

  Concurrent validity: when we compare results to another test with results that are known to be valid
• reliability 


  when the results of the study are consistent every time it is repeated

  a study can lack reliability if there are extraneous variables that affect the experiment or

  if the test used to measure the DV is itself unreliable
• internal reliability 


  how consistent the individual items on a test are with each other
• external reliability 


  when the results of a test are consistent every time its used
• split-half method 


  • measures the extent to which all parts of the test contribute equally to what is being measured
  • Comparing two halves of a test, questionnaire, or interview. We can test this by having participants sit the two halves of the task separately and seeing if their scores correlate (i.e: each person scores similarly in both tasks). To improve this we can remove and/or change questions to improve correlation.
• inter-rater reliability 


  extent to which two or more observers are observing and recording behaviour in the same way. We can assess it by comparing the results of two researchers on the same thing and see how well they correlate. We can improve inter rater reliability with training, practice in a pilot study, better operationalisation, and ensuring that each researcher has the same ability to see the subjects.
• test-retest reliability 


  Involves presenting the same participants with the same test or questionnaire on two separate occasions and seeing whether there is a positive correlation between the two. If correlation is poor then we should alter the task until it produces higher correlation
• factors that affect internal reliability 


  • Participant variables (i.e: personality, demand characteristics
  • Lack of experimental control (this covers order and Investigator effects)
  • Situational variables (i.e: time of day and temperature)
  • Researcher bias (i.e: lack of objectivity)
• content validity 


  type of internal validity

  The extent to which the questions/measurements in the study measure what we think we are measuring rather than some other fact
• face validity 


  type of internal validity

  • Simple way of assessing whether a test measures what it claims to measure which is concerned with face value – e.g. does an IQ test look like it tests intelligence.
• ecological validity 


  type of external validity

  • The extent to which the findings of a research study are able to be generalized to real-life settings
• mundane realism


  type of external validity

  • is the task similar to those encountered in real life
• population validity 


  type of external validity

  • Whether the research can be generalised to other people/populations.
• temporal validity 


  type of external validity

  • Refers to how likely it is that the time period when a study was conducted has influenced the findings and whether they can be generalised to other periods in time
• concurrent validity


  type of external validity

  • Comparing a new test with another test of the same thing to see if they produce similar results. If they do then the new test has concurrent validity
• external validity 


  Whether it is possible to generalise the results beyond the experimental setting (i.e: onto the wider population).

  External validity is improved by replicating the study in new situations or groups, and via use of field studies and naturalistic observation