test administration

• Flynn Effect


  Progressive rise in intelligence score that is expected to occur on a normed intelligence test from the date when the test was first normed
• Flynn Effect


  • Gradual increase in the general intelligence among newborns
• Frog Pond Effect


  Theory that individuals evaluate themselves as worse when in a group of high-performing individuals
• Culture-Free


  Attempt to eliminate culture so nature can be isolated
• It is impossible to develop a culture-free test because culture is evident in its influence since birth or an individual and the interaction between nature and nurture is cumulative and not relative
• Culture Fair


  Minimize the influence of culture with regard to various aspects of the evaluation procedures
• Types of culture fairness

  • Fair to all
  • Fair to some cultures
  • Fair only to one culture
• Culture Loading


  The extent to which a test incorporates the vocabulary concepts traditions, knowledge etc. with particular culture
• Classical Test Theory (True Score Theory)

  Score on ability tests is presumed to reflect not only the testtaker's true score on the ability being measured but also the error
• Error


  Refers to the component of the observed test score that does not have to do with the testtaker's ability
• Errors of measurement are random
• The greater number of items, the higher the reliability
• Factors that contribute to inconsistency


  • Characteristics of the individual
  • Characteristics of the test
  • Characteristics of the situation
• Error Variance


  Variance from irrelevant random sources
• Measurement Error


  All of the factors associated with the process of measuring some variable, other than the variable being measured
• Difference between observed score and true score

  Measurement Error
• Types of Measurement Error

  • Positive (can increase one's score)
  • Negative (decrease one's score)
• Sources of Error Variance

  • Item Sampling/Content Sampling
  • Test Administration
  • Test Scoring and Interpretation
• Random Error


  Source of error in measuring a targeted variable caused by unpredictable fluctuations and inconsistencies of other variables in measurement process (e.g., noise, temperature, weather)
• Systematic Error


  Source of error in a measuring a variable that is typically constant or proportionate to what is presumed to be the true values of the variable being measured
• Systematic error has a consistent effect on the true score, the SD does not change, the mean does
• Error variance may increase or decrease a test score by varying amounts, consistency of test score, and thus, the reliability can be affected
• Test-Retest Reliability


  Error: Time Sampling
• The longer the time passes, the greater likelihood that the reliability coefficient would be insignificant
• Carryover Effects


  Happened when the test-retest interval is short, wherein the second test is influenced by the first test because they remember or practiced the previous test = inflated correlation/overestimation of reliability
• Practice Effect


  Scores on the second session are higher due to their experience of the first session of testing
• Test-retest with longer interval might be affected of other extreme factors, thus, resulting to low correlation
• Target time for next administration: at least two weeks
• Parallel Forms/Alternate Forms Reliability


  Error: Item Sampling (Immediate), Item Sampling changes over time (delayed)
• Counterbalancing


  Technique to avoid carryover effects for parallel forms, by using different sequence for groups
• Main problem with parallel forms: difference between the two tests
• Test scores may be affected by motivation, fatigue, or intervening events
• Create a large set of questions that address the same construct then randomly divide the questions into two sets
• Internal Consistency (Inter-Item Reliability)


  Error: Item Sampling Homogeneity
• Split-Half Reliability


  Error: Item sample: Nature of Split
• Inter-Scorer Reliability


  Error: Scorer Differences
• Standard Error of Measurement

  Provide a measure of the precision of an observed test score
• Standard deviation of errors


  Basic measure of error
• Standard Error of Measurement

  Index of the amount of inconsistent or the amount of the expected error in an individual's score
• Higher reliability, lower SEM