UTILITY

• Utility


  Usefulness or practical value of testing to improve efficiency
• Utility can tell us something about the practical value of the information derived from scores on the test
• Utility


  Helps us make better decisions
• Higher criterion-related validity


  Higher utility
• Cost


  Disadvantages, losses, or expenses both economic and noneconomic terms
• Benefit


  Profits, gains or advantages
• The cost of test administration can be well worth it if the results is certain noneconomic benefits
• Utility Analysis


  Family of techniques that entail a cost-benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment
• Expectancy table


  Provide an indication that a testtaker will score within some interval of scores on a criterion measure – passing, acceptable, failing
• Expectancy table might indicate future behaviors, then if successful, the test is working as it should
• Taylor-Russel Tables


  Provide an estimate of the extent to which inclusion of a particular test in the selection system will improve selection
• Selection Ratio


  Numerical value that reflects the relationship between the number of people to be hired and the number of people available to be hired
• Base Rate


  Percentage of people hired under the existing system for a particular position
• One limitation of Taylor-Russel Tables is that the relationship between the predictor (test) and criterion must be linear
• Naylor-Shine Tables


  Entails obtaining the difference between the means of the selected and unselected groups to derive an index of what the test is adding to already established procedures
• Brogden-Cronbach-Gleser Formula


  Used to calculate the dollar amount of a utility gain resulting from the use of a particular selection instrument
• Utility Gain


  Estimate of the benefit of using a particular test
• Productivity Gains


  An estimated increase in work output
• High performing applicants may have been offered in other companies as well
• The more complex the job, the more people differ on how well or poorly they do that job
• Cut Score


  Reference point derived as a result of a judgement and used to divide a set of data into two or more classifications
• Relative Cut Score


  Reference point based on norm-related considerations (norm-referenced)
• Fixed Cut Scores


  Set with reference to a judgement concerning minimum level of proficiency required
• Multiple Cut Scores


  Refers to the use of two or more cut scores with reference to one predictor for the purpose of categorization
• Multiple Hurdle


  Multi-stage selection process, a cut score is in place for each predictor
• Compensatory Model of Selection

  Assumption that high scores on one attribute can compensate for lower scores
• Angoff Method


  Setting fixed cut scores, low interrater reliability
• Known Groups Method


  Collection of data on the predictor of interest from group known to possess and not possess a trait of interest, the determination of where to set cutoff score is inherently affected by the composition of contrasting groups
• IRT-Based Methods


  Cut scores are typically set based on testtaker's performance across all the items on the test, Item-Mapping Method: arrangement of items in histogram, with each column containing items with deemed to be equivalent value, Bookmark Method: expert places "bookmark" between the two pages that are deemed to separate testtakers who have acquired the minimal knowledge, skills, and/or abilities from those who are not
• Method of Predictive Yield

  Took into account the number of positions to be filled, projections regarding the likelihood of offer acceptance, and the distribution of applicant scores
• Discriminant Analysis


  Shed light on the relationship between identified variables and two naturally occurring groups