Chapter 7

• Refers to the usefulness or practical value of testing to improve efficiency.


  Utility
• Factors that Affect a Test's Utility:


  Psychometric soundness

  Costs

  Benefits
• A family of technique that entail a cost benefit analysis designed to yield information relevant to a division about the usefulness and/or practical value of a tool assessment.

  Utility Analysis
• How Is a Utility Analysis Conducted?


  Expectancy Data

  Brodgen Cronbach Gleser formula
• Estimate of the benefit of using a particular test or selection method.

  Utility gain
• Estimated increase in work output.


  Productivity gain
• Some Practical Considerations When Conducting Utility Analyses


  The pool of job applications

  The complexity of the job

  The cut score in use
• A reference point derived as a result of judgment and used to divide a set of data into two or more classifications.

  Cut score
• A reference point that is set based on norm-related considerations rather than on the relationship of test scores to a criterion.

  Relative cut score
• Cut score is set with reference to the performance of a group (or some target segment of a group.)


  Norm-referenced cut score
• Cut score that is typically set with reference to judgment concerning a minimum level of proficiency required to be included in a particular classification.

  Fixed cut score
• Refers to the use of two or more cut scores with reference to one predictor for the purpose of categorizing test takers.

  Multiple cut scores
• A cut score in place for each predictor used.

  Multiple hurdle
• Assumption is made that high scores on one attribute can compensate for low scores on another attribute.


  Compensatory model of selection
• Method for Setting Cut Scores:


  Angoff Method

  Known Groups Method

  IRT-Based Method

  Item-mapping Method

  Book Mark Method

  Other method: Discriminant analysis
• In this method judgments of experts are averaged.

  Angoff Method
• Collection of data on the predictor of interest from group known to possess and not to possess trait, attribute, or ability.

  Known Groups Method
• This is based on test taker's performance across all items on a test.

  IRT-Based Method
• Test items are listed, one per page, in ascending level of difficulty.

  Book-Mark Method
• Family of statistical techniques used to shed light on the relationship between certain variables and two or more naturally occurring groups.

  Discriminant analysis