6 Validity

• Validity


  The agreement between a test score or measure and the quality it is believed to measure
• Face Validity


  The mere appearance that a test has validity
• Face Validity


  • A scale to measure anxiety might include items such as "My stomach gets upset when I think about taking tests" and "My heart starts pounding fast whenever I think about all of the things I need to get done"
• Content Validity


  Considers the adequacy of representation of the conceptual domain the test is designed to cover
• Criterion-Related Validity


  Tells us just how well a test corresponds with a particular criterion
• Criterion


  The standard against which the test is compared
• Characteristics of a criterion

  • Relevant
  • Valid and reliable
  • Uncontaminated
• Predictive Validity


  Scores on a test can predict future behavior or scores on another test taken in the future
• Concurrent Validity


  The extent to which test scores can be used to estimate an individual's current standing on a criterion
• Validity Coefficient


  The relationship between a test and a criterion, usually expressed as a correlation, that tells the extent to which the test is valid for making statements about the criterion
• A validity coefficient is statistically significant if the chances of obtaining its value by chance alone are quite small, usually less than 5 in 100
• Construct-Related Validity


  Evidence established through a series of activities in which a researcher simultaneously defines some construct and develops the instrumentation to measure it
• Evidence of Homogeneity

  • Subtest scores are correlated to the total score
  • Coefficient alpha may be used as homogeneity evidence
  • Spearman Rho can be used to correlate an item to another item
  • Pearson or point biserial can be used to correlate an item to the total test score (inter-item correlation)
• Evidence of Changes with Age

  • Some variables or constructs are expected to change with age
• Evidence of Pretest-Posttest Changes

  • Difference of score from pretest and posttest of a defined construct after careful manipulation would provide validity
• Evidence from Distinct Groups

  • Method of contrasted groups
  • T-test can be used to test the difference of groups
• Convergent Evidence


  • A measure correlates well with other tests believed to measure the same construct
• Discriminant Evidence


  • A demonstration of uniqueness or the index taps something other than the tests used in the convergent evidence studies
• Factor Analysis

  • Exploratory factor analysis - entails "estimating, or extracting factors; deciding how many factors to retain; and rotating factors to an interpretable orientation"
  • Confirmatory factor analysis - researchers test the degree to which a hypothetical model fits the actual data
• Test Bias


  A factor inherent in a test that systematically prevents accurate, impartial measurement
• Types of Test Bias

  • Severity error/Strictness error
  • Leniency error/Generosity error
  • Central tendency error
  • Proximity error (order-effect)
  • Primacy effect
  • Contrast effect
  • Recency effect
  • Halo effect
  • Impression management
  • Acquiescence
  • Non-acquiescence
  • Faking-Good
  • Faking-Bad
• Test Fairness


  The extent to which a test is used in an impartial, just, and equitable way
• Reliability


  Consistency of measurement
• Validity


  Accuracy of measurement
• Attempting to define the validity of a test will be futile if the test is not reliable
• A construct is defined as something built by mental synthesis.
• Construct underrepresentation describes the failure to capture important components of a construct
• Construct-irrelevant variance occurs when scores are influenced by factors irrelevant to the construct.
• FACTOR ANALYSIS 


  A class of mathematical procedures designed to identify factors or specific variables that are typically attributes, characteristics, or dimensions on which people may differ
• IOW, we can have reliability without validity. However, it is logically impossible to demonstrate that an unreliable test is valid.