Validity and Reliability

• What does validity assess in research?

  Whether a measure truly measures what it claims
• What does validity ensure about research?

  That research is true and accurate
• What does validity ensure the research does?

  What it intended to do
• What does reliability assess in research?

  How consistent the research is
• What does reliability mean about research results?

  The results are stable
• If you repeated a reliable study, what would you expect?

  To get the same results
• What does internal validity assess?

  Effects are due to IV manipulation
• What does internal validity exclude as a cause?

  Some other factor
• What is the primary question regarding internal validity?

  Did the IV cause the change?
• What does external validity refer to?

  Generalization to other situations
• What are the two types of external validity?

  • Ecological validity
  • Population validity
• What does ecological validity assess?

  Representation of the outside world
• If research is conducted in a different setting, what should happen according to ecological validity?

  The results should be the same
• What does population validity address?

  Representation of the target population
• What happens if the sample doesn't represent the target population?

  Low population validity
• What issues affect internal validity and how to deal with them?

  • Extraneous variables: Run a pilot study to identify and control them.
  • Demand characteristics: Use a single-blind procedure and "red herring" questions.
  • Researcher bias: Follow standardized procedures and use a double-blind technique.
  • Social desirability bias: Ensure confidentiality and do not ask for names.
• What is the purpose of a pilot study?

  To identify extraneous variables
• What should you do to make sure participants complete the study under the same conditions?

  Control possible extraneous variables
• What is the purpose of single-blind procedure?

  To hide aims of study
• What is the purpose of 'red herring' questions?

  To prevent guessing study aims
• What is the purpose of following standardized procedures?

  To reduce researcher bias
• What does a double-blind technique involve?

  Researcher unaware of study aims
• What can ensure confidentiality of participants?

  Not asking for names
• What does reliability assess in research?

  How consistent the research is
• What does reliability mean about research results?

  The results are stable
• If you repeated a reliable study, what would you expect?

  To get the same results
• What does internal reliability assess?

  Consistency across items within a test
• What should a questionnaire measuring hardiness consistently measure?

  Hardiness to the same extent
• What does external reliability refer to?

  Consistency if repeated with same person
• What should happen if an IQ test is repeated on the same person?

  Get the same result
• What issues affect reliability and how to deal with them?

  • Operationalisation of variables: Clearly define and measure variables.
  • Lack of standardized procedures: Use standardized instructions and procedures.
  • Low inter-observer reliability: Train observers, operationalize behavior categories, and pilot observation records.
• What makes replication impossible?

  Poor variable operationalization
• What affects consistency due to poor instructions?

  A lack of standardized procedures
• What is low inter-observer reliability?

  Observers don't get the same results
• How can a test's reliability be ensured?

  Pilot the test first
• What causes issues with external reliability?

  The way research is explained
• How do you reduce external reliability issues?

  Use standardized instructions
• How should participants be treated to reduce external validity?

  In exactly the same way
• What should you do to ensure interobserver reliability?

  Train the observers beforehand
• What is a pilot study?

  A pre-study