RELIABILITY

• Reliability


  Dependability or consistency of the instrument or scores obtained by the same person when re-examined with the same test on different occasions, or with different sets of equivalent items
• Test may be reliable in one context, but unreliable in another
• Estimate the range of possible random fluctuations that can be expected in an individual's score

  Free from errors
• More number of items

  Higher reliability
• Minimizing error


  Using only representative sample to obtain an observed score
• True score cannot be found
• Reliability Coefficient


  Index of reliability, a proportion that indicates the ratio between the true score variance on a test and the total variance
• Classical Test Theory (True Score Theory)

  Score on a ability tests is presumed to reflect not only the testtaker's true score on the ability being measured but also the error
• Error


  Refers to the component of the observed test score that does not have to do with the testtaker's ability
• Errors of measurement are random
• When you average all the observed scores obtained over a period of time, then the result would be closest to the true score
• The greater number of items


  The higher the reliability
• Factors that contribute to consistency

  • Stable attributes
• Factors that contribute to inconsistency


  • Characteristics of the individual, test, or situation, which have nothing to do with the attribute being measured, but still affect the scores
• Goals of Reliability


  • Estimate errors
  • Devise techniques to improve testing and reduce errors
• Variance


  Useful in describing sources of test score variability
• True Variance


  Variance from true differences
• Error Variance


  Variance from irrelevant random sources
• Measurement Error


  All of the factors associated with the process of measuring some variable, other than the variable being measured
• Difference between the observed score and the true score

  • Positive: can increase one's score
  • Negative: decrease one's score
• Sources of Error Variance

  • Item Sampling/Content Sampling
  • Test Administration
  • Test Scoring and Interpretation
• Item Sampling/Content Sampling


  Refer to variation among items within a test as well as to variation among items between tests
• Test Administration


  Testtaker's motivation or attention, environment, etc.
• Test Scoring and Interpretation


  May employ objective-type items amenable to computer scoring of well-documented reliability
• Random Error


  Source of error in measuring a targeted variable caused by unpredictable fluctuations and inconsistencies of other variables in measurement process (e.g., noise, temperature, weather)
• Systematic Error


  Source of error in a measuring a variable that is typically constant or proportionate to what is presumed to be the true values of the variable being measured
• Systematic error has consistent effect on the true score, the SD does not change, the mean does
• Reliability


  Refers to the proportion of total variance attributed to true variance
• The greater the proportion of the total variance attributed to true variance, the more reliable the test
• Error variance may increase or decrease a test score by varying amounts, consistency of test score, and thus, the reliability can be affected
• Test-Retest Reliability


  Error: Time Sampling
• Time sampling reliability


  An estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the test
• Appropriate when evaluating the reliability of a test that purports to measure an enduring and stable attribute such as personality trait
• The longer the time passes, the greater likelihood that the reliability coefficient would be insignificant
• Carryover Effects


  Happened when the test-retest interval is short, wherein the second test is influenced by the first test because they remember or practiced the previous test = inflated correlation/overestimation of reliability
• Practice Effect


  Scores on the second session are higher due to their experience of the first session of testing
• Test-retest with longer interval might be affected of other extreme factors, thus, resulting to low correlation
• Lower correlation = poor reliability
• Mortality


  Problems in absences in second session (just remove the first tests of the absents)
• Coefficient of Stability


  Statistical tool: Pearson R, Spearman Rho