6 Validity
Cards (30)
- ValidityThe agreement between a test score or measure and the quality it is believed to measure
- Face ValidityThe mere appearance that a test has validity
- Face Validity
- A scale to measure anxiety might include items such as "My stomach gets upset when I think about taking tests" and "My heart starts pounding fast whenever I think about all of the things I need to get done"
- Content ValidityConsiders the adequacy of representation of the conceptual domain the test is designed to cover
- Criterion-Related ValidityTells us just how well a test corresponds with a particular criterion
- CriterionThe standard against which the test is compared
- Characteristics of a criterion
- Relevant
- Valid and reliable
- Uncontaminated
- Predictive ValidityScores on a test can predict future behavior or scores on another test taken in the future
- Concurrent ValidityThe extent to which test scores can be used to estimate an individual's current standing on a criterion
- Validity CoefficientThe relationship between a test and a criterion, usually expressed as a correlation, that tells the extent to which the test is valid for making statements about the criterion
- A validity coefficient is statistically significant if the chances of obtaining its value by chance alone are quite small, usually less than 5 in 100
- Construct-Related ValidityEvidence established through a series of activities in which a researcher simultaneously defines some construct and develops the instrumentation to measure it
- Evidence of Homogeneity
- Subtest scores are correlated to the total score
- Coefficient alpha may be used as homogeneity evidence
- Spearman Rho can be used to correlate an item to another item
- Pearson or point biserial can be used to correlate an item to the total test score (inter-item correlation)
- Evidence of Changes with Age
- Some variables or constructs are expected to change with age
- Evidence of Pretest-Posttest Changes
- Difference of score from pretest and posttest of a defined construct after careful manipulation would provide validity
- Evidence from Distinct Groups
- Method of contrasted groups
- T-test can be used to test the difference of groups
- Convergent Evidence
- A measure correlates well with other tests believed to measure the same construct
- Discriminant Evidence
- A demonstration of uniqueness or the index taps something other than the tests used in the convergent evidence studies
- Factor Analysis
- Exploratory factor analysis - entails "estimating, or extracting factors; deciding how many factors to retain; and rotating factors to an interpretable orientation"
- Confirmatory factor analysis - researchers test the degree to which a hypothetical model fits the actual data
- Test BiasA factor inherent in a test that systematically prevents accurate, impartial measurement
- Types of Test Bias
- Severity error/Strictness error
- Leniency error/Generosity error
- Central tendency error
- Proximity error (order-effect)
- Primacy effect
- Contrast effect
- Recency effect
- Halo effect
- Impression management
- Acquiescence
- Non-acquiescence
- Faking-Good
- Faking-Bad
- Test FairnessThe extent to which a test is used in an impartial, just, and equitable way
- ReliabilityConsistency of measurement
- ValidityAccuracy of measurement
- Attempting to define the validity of a test will be futile if the test is not reliable
- A construct is defined as something built by mental synthesis.
- Construct underrepresentation describes the failure to capture important components of a construct
- Construct-irrelevant variance occurs when scores are influenced by factors irrelevant to the construct.
- FACTOR ANALYSISA class of mathematical procedures designed to identify factors or specific variables that are typically attributes, characteristics, or dimensions on which people may differ
- IOW, we can have reliability without validity. However, it is logically impossible to demonstrate that an unreliable test is valid.