Test Development

• Test Development


  1. Test Conceptualization

  2. Item Development

  3. Test Construction

  4. Test Tryout

  5. Test Revision

  
• Test Conceptualization


  Defining what the test is designed to measure, its objectives, need, target users, content, administration, format, training required, response types, benefits, and potential for harm

  
• Norm-referenced


  Typically insufficient and inappropriate when knowledge of mastery is required
• Criterion-referenced


  Items were designed to assess if the test taker has met a certain criteria
• Pilot work, pilot study, and pilot research

  Preliminary research surrounding the creation of a prototype of the test, not necessary for developing teacher-made tests for classroom use
• Scaling


  Types of scales (ordinal, nominal) and scaling methods based on factors
• Item Writing


  Determining content coverage, item formats, and total number of items
• Item Types


  • True-False
  • Multiple-Choice
  • Matching
  • Short Answer or Completion
• Scoring Items


  Class or category scoring, ipsative scoring (forced-choice measurement)
• Test Tryout


  1. Item Analysis (item difficulty index, item discrimination index, analysis of item alternatives, item reliability index, item validity index)

  2. Other Considerations (guessing, item fairness, speed tests, qualitative item analysis)
• Test Revision


  1. Test conceptualization phase

  2. Test construction phase

  3. Test tryout

  4. Item analysis

  5. Test revisions

  
• Cross-validation


  Revalidation of a test on a sample of test takers other than those on whom test performance was originally found to be a valid predictor of some criterion
• Co-validation


  A test validation process conducted on two or more tests using the same sample of test takers
• Quality assurance during test revision

  • Employ examiners with experience on testing members of the targeted population
  • Anchor protocol
• Use of IRT in building and revising tests


  • Evaluating the properties of existing tests and guiding test revision
  • Determining measurement equivalence across test taker populations
  • Developing item banks