validity and reliability

• reliability is the extent to which a test produces a consistent finding every time
• if an experiment or study is reliable we expect to find the same results carried out on another day
• assess reliability by using:

  • questionnaires and psychological testing
  • observations
• interobserver reliability is the extent to which there is agreement between 2 or more observers involved in observations independently and compare their results to ensure no bias
• test retest reliability is administrating same test/questionnaire to same person on different occasions - if reliable results obtained should be the same
• observations are measured by correlating the observation of 2 or more observers

  • strong correlation = reliability is good
• when using observations to test reliability there must be sufficient time between the test and retest (1-2 weeks) to ensure participants cannot recall their answers
• questionnaires can be improved by:

  • making sure items on scale/questions asked are clear so cannot be interpreted
  • if open ended questions being asked they can be changes to closed
  • try use same interviewer each time so that they know not to ask leading questions
• observations can be improved by

  • ensuring all behavioural categories and criteria are clearly defined
  • ensure observers are sufficiently trained and practiced in using behavioural categories as they can respond more quickly
• experiments can be improved by

  • procedures repeated are exactly the same so they are standardised
  • careful in controlling effects of confounding variables and control research
• validity is making sure the data we collect in study represents reality
• validity can be split into concerns about what is going on inside (internal validity) and what is going on outside (external validity)
• internal validity is what goes on within the study
• internal validity is a measure of internal consistency of experiment and to the extent to which it can be said IV causes change to DV
• external validity is what goes on after the study
• external validity is the measure of extend to which results from experiment can be generalised beyond context of experiment
• factors affecting internal validity include

  • extraneous variables
  • confounding variables
  • demand characteristics
  • investigator effects
  • social desirability bias
  • poor operalisation
• assessing validity can be done through face and concurrent
• using concurrent to assess validity

  • extend to which psychological measure relates to existing, well established and similar measure
• if the low concurrent validity is low then researchers should move questions which may seem irrelevant
• if scale or measure is judged to have poor face validity then questions should be revised so they relate more obviously to the topic
• by controlling confounding variables you can impove validity
• reducing the possibility of researcher bias (ensuing tasks is objective and little or no interpretation) is another way of improving validity
• eliminating investigating effects (wearing same clothing, same scripts) can also help to improve validity of experiment
• to improve external validity you should use large sample (10% of target population) in order to be able to generalise your random sample