Psych Assessment 1

• Item analysis


  Conducted to ensure the objectivity of the test items and the test itself. The reliability and validity of a test can be built into the instrument in advance through item analysis.
• Methods of item analysis in ability tests

  • The U-L index method
  • Point-Biserial Correlation (rpb) Method
  • Phi Coefficient (Φ)
• The U-L index method

  1. Score the test

  2. Arrange the papers from highest to lowest

  3. Separate the top 27% and bottom 27% of the examinees

  4. Prepare a tally sheet

  5. Convert the tallies to frequencies and then to proportions

  6. Compute for the Difficulty Index of each item

  7. Compute the Discrimination Index of each item (Ds) and evaluate them
• Item Difficulty (Df)


  Refers to the proportion or percentage of examinees who correctly answered the item. The easier the item, the larger the proportion or percentage.
• Difficulty index evaluation


  • Below .20 = Difficult
  • 0.20 - 0.80 = Good Item
  • Above 0.80 = Easy
• Item Discrimination (Ds)


  Refers to the degree to which an item differentiates correctly among examinees in the behavior that the test is designed to measure.
• Discrimination index evaluation


  • 0.40 - Above = Very Good Item - RETAIN
  • 0.30 - 0.39 = Good Item - RETAIN
  • 0.20 - 0.29 = Marginal Item - REVISE
  • Below 0.20 = Poor Item - REJECT
• Point-Biserial Correlation (rpb)


  Used to analyze the test items that are scored as 1 or 0. Involves correlating total score (X) and item score (Y).
• Point-Biserial Correlation (rpb) evaluation

  • rpb = 0.30 - Higher = Good item/Retain
  • rpb = Below 0.30 = Poor item/Reject
• Phi Coefficient (Φ)


  The coefficient is computed from a four-fold table and is based on the number of examinees passing and failing an item in the U and L criterion groups.
• Phi Coefficient (Φ) evaluation

  • Φ = 0.30 - Higher = Good item/Retain
  • Φ = Below 0.30 = Poor item/Reject
• Analysis of distracters is also a method of item analysis
• 𝒅 𝒊𝒕𝒆𝒎/��𝒆𝒕𝒂𝒊𝒏


  Below 0.30 = 𝑷𝒐𝒐𝒓 𝒊𝒕𝒆𝒎/𝑹𝒆𝒋𝒆𝒄𝒕
• Solve Example


  1. Determine number of examinees

  2. Determine upper group (top 27%)

  3. Determine lower group (lowest 27%)

  4. Evaluate items 1, 2, and 3
• Phi Coefficient (Φ)


  Measure of the relationship between two binary variables
• Φ = (ad - bc) / √(klmn)
• Φ = 0.58 for Item 1
• Φ = 0.08 for Item 2
• Φ = 0.19 for Item 3
• Distractors


  • Good distractors are selected more frequently by members of the lower group
  • A distractor not chosen by any member of the upper and lower group must be changed or revised
  • If a distractor is more popular than the correct answer, the answers might be keyed incorrectly or contains ambiguities
• For Item 1, Distractor B should be replaced and Distractor A may be revised
• For Item 2, Distractor A is more popular than the correct answer (B); the item may be keyed incorrectly or contains ambiguities
• For Item 2, Distractor D may be revised to make it more attractive to the lower group
• Pearson Product-Moment Correlation (r)


  Method used for tests of continuous scaling with three (3) or more scale points like the Likert Scales or Bipolar scales to determine if an item contributes to the measurement of the trait being appraised
• r ≥ 0.30 is reflective of a good item
• r = 0.76 for Item 1
• r = 0.86 for Item 2
• r = 0.45 for Item 3
• r = 0.80 for Item 4
• r = 0.58 for Item 5
• who develop the u-l index method