PSYCH ASSESS FINALS

• Utility


  Usefulness or practical value of testing to improve efficiency
• Testing


  Refers to anything from a single test to a large-scale testing program that employs a battery of tests
• Factors That Affect a Test's Utility

  • Psychometric Soundness
  • Costs
  • Benefits
• Psychometric Soundness


  The reliability and validity of a test
• Index of Reliability


  About how consistently a test measures what it measures
• Index of Validity


  Whether a test measures what it purports to measure
• Index of Utility


  The practical value of the information derived from scores on the test
• Test scores have utility

  If their use in a particular situation helps us to make better decisions—better, that is, in the sense of being more cost-effective
• Costs of testing


  • Payment to professional personnel and staff associated with test administration, scoring, and interpretation
  • Facility rental, mortgage, and/or other charges related to the usage of the test facility
  • Insurance, legal, accounting, licensing, and other routine costs of doing business
• Potential Benefits


  • Increase in the quality of workers' performance
  • Increase in the quantity of workers' performance
  • Decrease in the time needed to train workers
  • Reduction in the number of accidents
  • Reduction in worker turnover
• Utility Analysis


  A family of techniques that entail a cost-benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment
• Expectancy Data / Expectancy Table


  An indication of the likelihood that a testtaker will score within some interval of scores on a criterion measure—an interval that may be categorized as "passing," "acceptable," or "failing"
• Top-down selection


  A process of awarding available positions to applicants whereby the highest scorer is awarded the first position, the next highest scorer the next position, and so forth until all positions are filled
• Hit


  Accurately identifies as possessing or exhibiting a particular trait, ability, behavior, or attribute
• Miss


  Inaccurately identifies as possessing or exhibiting a particular trait, ability, behavior, or attribute
• Hit Rate


  Accurately identifies as possessing or exhibiting a particular trait, ability, behavior, or attribute
• Miss Rate


  Inaccurately identifies as possessing or exhibiting a particular trait, ability, behavior, or attribute
• False Positive


  A specific type of miss whereby an assessment tool falsely indicates that the testtaker possesses or exhibits a particular trait, ability, behavior, or attribute
• False Negative


  A specific type of miss whereby an assessment tool falsely indicates that the testtaker does not possess or exhibit a particular trait, ability, behavior, or attribute
• Utility Analysis Made Easy

  A program that contains definitions for a wealth of utility-related terms and also provides the tools for automatically creating computer-generated, utility-related tables and graphs
• Taylor-Russell tables


  Tables that provide an estimate of the extent to which the inclusion of a particular test in the selection system will improve selection
• Selection Ratio


  A numerical value that reflects the relationship between the number of people to be hired and the number of people available to be hired
• Base Rate


  Refers to the percentage of people hired under the existing system for a particular position
• Naylor-Shine tables


  Obtaining the difference between the means of the selected and unselected groups to derive an index of what the test (or some other tool of assessment) is adding to already established procedures
• Brogden-Cronbach-Gleser formula


  Used to calculate the dollar amount of a utility gain resulting from the use of a particular selection instrument under specified conditions
• Utility Gain


  Refers to an estimate of the benefit (monetary or otherwise) of using a particular test or selection method
• Productivity Gain


  Refers to an estimated increase in work output
• Fixed Cut Score


  Defines a reference point, also known as absolute cut scores
• Multiple Cut Scores


  Refers to the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers
• Multiple Hurdle


  May be thought of as one collective element of a multistage decision-making process in which the achievement of a particular cut score on one test is necessary in order to advance to the next stage of evaluation
• Compensatory model of selection

  An assumption is made that high scores on one attribute can, in fact, "balance out" or compensate for low scores on another attribute
• Method of Cutting Scores

  • Angoff method
  • Known Groups Method
  • IRT Based Methods / Item Response Theory
  • Item-mapping method
  • Bookmark method
  • Method of predictive yield
  • Discriminant Analysis / Discriminant Function Analysis
• Test Development


  An umbrella term for all that goes into the process of creating a test
• Stages of test development

  1. Test conceptualization

  2. Test construction

  3. Test tryout

  4. Item analysis

  5. Test revision
• Test Conceptualization


  The stage where test construction begins
• Test Construction


  A stage in the process of test development that entails writing test items (or re-writing or revising existing items), as well as formatting items, setting scoring rules, and otherwise designing and building a test
• Test Tryout


  Once a preliminary form of the test has been developed, it is administered to a representative sample of testtakers under conditions that simulate the conditions that the final version of the test will be administered
• Item Analysis


  Employed to assist in making judgments about which items are good as they are, which items need to be revised, and which items should be discarded
• Test Revision


  Refers to action taken to modify a test's content or format for the purpose of improving the test's effectiveness as a tool of measurement
• Norm Referenced


  Any assessment in which scores are interpreted by comparison with a norm, generally the average score obtained by members of a specified group