Reliability and validity

• What is reliability in testing?

  Consistency of test results over time
• What do we expect from a reliable test?

  Same results on different occasions
• What are the ways to assess reliability?

  • Test-retest method
  • Inter-observer reliability
  • Content analysis

  
• What is the test-retest method?

  Same test given on different occasions
• What should happen if a test is reliable?

  Results should be similar each time
• What types of tests mainly use the test-retest method?

  Questionnaires and psychological tests
• How is reliability assessed in questionnaires?

  By correlating two sets of scores
• What indicates good reliability in correlation?

  Correlation coefficient exceeds +.80
• What is inter-observer reliability?

  Observers compare results after watching the same event
• What is the purpose of content analysis?

  To analyze qualitative data quantitatively
• What are coding units in content analysis?

  Units used to categorize qualitative data
• What is the first step in conducting content analysis?

  Identify the qualitative data to analyze
• How is reliability measured in test-retest and inter-observer reliability?

  By correlating two sets of scores
• What correlation coefficient indicates reliability?

  Exceeds +.80
• How can reliability be improved in questionnaires?

  Using the test-retest method

  
• What should be consistent in experiments to improve reliability?

  Procedures must be the same every time
• How should behavioral categories be defined in observations?

  Operationalized and non-overlapping
• What is the role of the interviewer in improving reliability?

  Use the same interviewer each time
• What is validity in psychological testing?

  Accuracy of the test results
• What are the two types of validity?

  Internal and external validity
• What are some problems with internal validity?

  Investigator effects and confounding variables
• What does external validity concern?

  Generalizability of findings to other contexts
• What is face validity?

  Test appears to measure what it claims
• What is concurrent validity?

  Comparison with a well-established test
• What is a control group used for in experiments?

  To ensure changes are due to the IV
• How can questionnaires improve validity?

  By including a lie scale in questions
• What is ecological validity in observations?

  Findings reflect natural behavior
• What enhances validity in qualitative research?

  Triangulation of different evidence sources
• What is interpretive validity?

  Researcher's interpretation matches participants' views
• Why is measuring IQ through head circumference not valid?

  It does not accurately measure intelligence
• What are the problems with internal validity?

  • Investigator effects
  • Demand characteristics
  • Confounding variables
  • Extraneous variables
  • Social desirability
  • Poorly operationalized behavioral categories
• What are the problems with external validity?

  • Population validity
  • Historical validity
  • Ecological validity
  • Mundane realism
• How can validity be improved in experiments?

  • Use a control group
  • Standardized procedures
  • Single-/double-blind methods
• How can validity be improved in questionnaires?

  • Include a lie scale
  • Assure confidentiality
• How can validity be improved in observations?

  • Use covert observations
  • Define clear behavioral categories
• How can validity be improved in qualitative research?

  • Demonstrate interpretive validity
  • Use triangulation of evidence
• What is the relationship between reliability and validity?

  • Reliable results can be inaccurate
  • Valid results must be reliable