3. Validity

    Cards (73)

    • Validity
      A judgment or estimate of how well a test measures what it supposed to measure
    • Validity
      Evidence about the appropriateness of inferences drawn from test scores
    • Validity
      Degree to which the measurement procedure measures the variables to measure
    • Inferences
      Logical result or deduction
    • Validity may diminish as the culture or times change
    • Validity
      • Predicts future performance
      • Measures appropriate domain
      • Measures appropriate characteristics
    • Validation
      The process of gathering and evaluating evidence about validity
    • Validation Studies
      Yield insights regarding a particular population of test-takers as compared to the norming sample described in a test manual
    • Internal Validity
      Degree of control among variables in the study (increased through random assignment)
    • External Validity
      Generalizability of the research results (increased through random selection)
    • Conceptual Validity

      Focuses on individual with their unique histories and behaviors
    • Conceptual Validity
      Means of evaluating and integrating test data so that the clinician's conclusions make accurate statements about the examinee
    • Face Validity
      A test appears to measure to the person being tested than to what the test actually measures
    • Content Validity
      Describes a judgement of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample
    • Content Validity
      Representativeness and relevance of the assessment instrument to the construct being measured
    • Content Validity
      When the proportion of the material covered by the test approximates the proportion of material covered in the course
    • Test Blueprint
      A plan regarding the types of information to be covered by the items, the number of items tapping each area of coverage, the organization of the items, and so forth
    • Content Validity is more logical than statistical
    • Content Validity
      Concerned with the extent to which the test is representative of defined body of content consisting the topics and processes
    • Evaluating Content Validity
      1. Panel of experts can review the test items and rate them in terms of how closely they match the objective or domain specification
      2. Examine if items are essential, useful and necessary
    • Construct underrepresentation
      Failure to capture important components of a construct
    • Construct-irrelevant variance

      Happens when scores are influenced by factors irrelevant to the construct
    • Lawshe
      Developed the formula of Content Validity Ratio
    • Low Content Validity Ratio (CVR)

      Recommended to remove or modify the items that have low CVR values to improve the overall content validity of the test
    • Zero CVR: exactly half of the experts rate the item as essential
    • Criterion Validity

      A judgement of how adequately a test score can be used to infer an individual's most probable standing on some measure of interest – the measure of interest being criterion
    • Criterion
      Standard on which a judgement or decision may be made
    • Criterion Validity
      Characteristics: relevant, valid, uncontaminated
    • Criterion Contamination
      Occurs when the criterion measure includes aspects of performance that are not part of the job or when the measure is affected by "construct-irrelevant" (Messick, 1989) factors that are not part of the criterion construct
    • Concurrent Validity

      If the test scores obtained at about the same time as the criterion measures are obtained; economically efficient
    • Predictive Validity

      Measures of the relationship between test scores and a criterion measure obtained at a future time
    • Incremental Validity

      The degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use; used to improve the domain
    • Construct Validity (Umbrella Validity)

      Covers all types of validity, logical and statistical, judgement about the appropriateness of inferences drawn from test scores regarding individual standing on variable called construct
    • Construct
      An informed, scientific idea developed or hypothesized to describe or explain behavior; unobservable, presupposed traits that may invoke to describe test behavior or criterion performance
    • Improving homogeneity of a test with dichotomous items
      Eliminating items that do not show significant correlation coefficients with total test scores
    • If high scorers on the entire test for some reason tended to get a particular item wrong while low scorers got it right, then the item is obviously not a good one
    • Some constructs lend themselves more readily than others to predictions of change over time
    • Method of Contrasted Groups
      Demonstrate that scores on the test vary in a predictable way as a function of membership in a group
    • If a test is a valid measure of a particular construct, then the scores from the group of people who does not have that construct would have different test scores than those who really possesses that construct
    • Convergent Evidence
      If scores on the test undergoing construct validation tend to highly correlated with another established, validated test that measures the same construct
    See similar decks