Validity 1

• Validity


  A judgment or estimate of how well a test measures what it purports to measure in a particular context
• Validation


  The process of gathering and evaluating evidence about validity
• Categories of validity


  • Content validity
  • Criterion-related validity
  • Construct validity
• Assessing validity


  1. Scrutinizing the test's content

  2. Relating scores obtained on the test to other test scores or other measures

  3. Executing a comprehensive analysis of how scores on the test relate to other test scores and measures, and how scores on the test can be understood within some theoretical framework for understanding the construct that the test was designed to measure
• Face validity


  Relates more to what a test appears to measure to the person being tested than to what the test actually measures
• Content validity


  A judgment of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample
• Test blueprint


  A plan regarding the types of information to be covered by the items, the number of items tapping each area of coverage, the organization of the items in the test
• Content validity ratio (CVR)

  Quantification of content validity

  
• Criterion-related validity


  A judgment of how adequately a test score can be used to infer an individual's most probable standing on some measure of interest—the measure of interest being the criterion

  
• Types of criterion-related validity

  • Concurrent validity
  • Predictive validity
• Validity coefficient


  A correlation coefficient that provides a measure of the relationship between test scores and scores on the criterion measure
• Incremental validity


  The degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use
• Decision theory and test utility

  • Base rate
  • Hit rate
  • Miss rate
  • False positive
  • False negative
• Construct validity


  A judgment about the appropriateness of inferences drawn from test scores regarding individual standings on a variable called a construct
• Evidence of construct validity

  • Evidence of homogeneity
  • Evidence of changes with age
  • Evidence of pretest–posttest changes
  • Evidence from distinct groups
  • Convergent evidence
  • Discriminant evidence
• Types of factor analysis

  • Exploratory factor analysis
  • Confirmatory factor analysis
• Bias


  A factor inherent in a test that systematically prevents accurate, impartial measurement
• Types of bias


  • Intercept bias
  • Slope bias
• Rating


  A numerical or verbal judgment (or both) that places a person or an attribute along a continuum identified by a scale
• Rating error


  A judgment resulting from the intentional or unintentional misuse of a rating scale
• Ranking


  A procedure that requires the rater to measure individuals against one another instead of against an absolute scale
• Halo effect


  A tendency to give a particular ratee a higher rating than he or she objectively deserves because of the rater's failure to discriminate among conceptually distinct and potentially independent aspects of a ratee's behavior
• Test Fairness


  The extent to which a test is used in an impartial, just, and equitable way
• Validity refers to whether a measure actually measures what it claims to measure.