Assessment of Learning

• Summative assessments occur at the end of an instructional unit or course and evaluate student performance against predetermined standards.
• Test is an instrument designed to measure any characteristic, quality, ability, knowledge or skill. It comprises items in the area it is designed to measure.
• Assessment is a process of gathering and organizing quantitative and qualitative data into an interpretative form to have a basis for judgment or decision making. It is a prerequisite to evaluation for it provides information for evaluation to take place.
• EVALUATION is the process of systematic information, analysis, appraisal or judgment of the worth of organized data as the basis for decision-making. It involves judgment about the desirability of changes to students.
• kinds of assessment - Traditional, Alternative, and Authentic
• Purpose of Assessment - Assessment for learning, Assessment of learning, and Assessment as Learning
• Assessment for Learning focuses on the students' learning process and how they learn. It was done before, and during the lecture. Placement, Formative, and Diagnostic are examples of assessment for learning.
• Assessment of Learning is the process of evaluating the effectiveness of the learning process. It is done after the lesson was taught. Example is summative test.
• Assessment as Learning - the process of evaluating the effectiveness of learning. It is done by the teachers to know if their learning methods are effective.
• Bloom's Taxonomy is a model of learning that describes the different levels of cognitive processing.
• Bloom's Taxonomy

  • Remembering
  • Understanding
  • Applying
  • Analyzing
  • Evaluating
  • Creating
• Multiple-choice items can be used to measure knowledge outcomes and various types of learning outcomes. They are most widely used for measuring knowledge, comprehension and application outcomes. The multiple-choice item provides the most useful format for measuring achievement at various levels of learning.
• Strengths of multiple-choice exams:

  • Learning outcomes from simple to complex can be measured
  • A broad sample of achievement can be measured
  • Highly structured and clear tasks are provided
  • Incorrect alternatives provide diagnostic information
  • Items can be written so that students must discriminate from options that vary in degree of correctness
  • Can cover a lot of material very efficiently (about one item per minute of testing time)
  • Scoring is easy, objective and reliable
  • Scores are less influenced by guessing than True-False items
  • Scores are more reliable than subjectively scored items (e.g. essays)
  • Item analysis can reveal how difficult each item was and how well it discriminated between the strong and weaker students in the class
  • Performance can be compared from class to class and year to year
• Limitations/Challenges of Multiple Choice Exams:

  • Constructing good items is time consuming
  • It is frequently difficult to find plausible distracters
  • Sometimes there is more than one defensible “correct” answer
  • Often focus on testing factual information and fails to test higher levels of cognitive skills
  • This item is ineffective for measuring some types of problem solving and the ability to organize and express ideas
  • Students can sometimes read more into the question than was intended
  • Scores can be influenced by reading ability
  • May encourage guessing
  • There is a lack of feedback on individual thought processes
  • Real-world problem solving differs from selecting a solution from a set of alternatives
• An age old principle of the teaching cycle

  
• TYPES OF ITEMS THAT STUDENTS PERCEIVED AS TRICKY

  1. The test writer’s intention appeared to deceive, confuse or mislead test takers.
  2. Trivial content was represented.
  3. The discrimination among options was too fine.
  4. Items had window dressing that was irrelevant to the problem.
  5. Multiple correct answers were possible.
  6. Principles were presented in ways that were not learned, thus deceiving students.
  7. Items were so highly ambiguous that even the best students had no idea about the right answer.
• Content Concerns:

  • Every item should reflect specific content and a single specific cognitive process
  • Base each item on important learning material; avoid trivial content
  • Use novel material to test understanding and higher-level learning
  • Keep item content independent of other test items
  • Avoid over-specific and over-general content
  • Avoid opinion-based and trick questions
• Format And Style:

  • Format items vertically
  • Edit items for correct grammar, punctuation, capitalization, and spelling
  • Proofread each item
  • Make directions as clear as possible
• Stem Construction:

  • Make the stem as brief as possible
  • Place the main idea in the stem, not in the choices
  • Avoid irrelevant information in the stem
  • Avoid negative words in the stem
• Option Development:

  • Key:
  • Ensure there is one and only one correct option
  • Vary the location of the right answer according to the number of options and assign the position of the correct answer randomly
  • Distracters:
  • Use three choices as a natural limit
  • Place options in logical or numerical order
  • Keep choices independent and non-overlapping
  • Keep choices homogenous in content and grammatical structure
  • Keep the length of choices about the same
  • Use "None of the above" sparingly
  • Avoid using "All of the above"
  • Avoid negative words like "not" or "except"
  • Make all distractors plausible
  • Avoid options that give clues to the right answer
• Format And Style:

  • Format items vertically instead of horizontally
  • Edit items for correct grammar, punctuation, capitalization, and spelling
  • Proofread each item
  • Make directions as clear as possible
• Stem Construction:

  • Make the stem as brief as possible
  • Place the main idea in the stem, not in the choices
  • Avoid irrelevant information
  • Avoid negative words in the stem
• Option Development:

  • Ensure there is one and only one correct option
  • Vary the location of the right answer according to the number of options
  • Use three distractors
  • Keep choices independent, homogenous, and about the same length
• WAYS IN ESTABLISHING VALIDITY 

  1. FACE VALIDITY
  2. CONTENT VALIDITY
  3. CRITERION-RELATED VALIDITY
  4. concurrent validity
  5. predictive validity
  6. CONSTRUCT VALIDITY
  7. convergent validity
  8. divergent validity
• METHODS IN ESTABLISHING RELIABILITY 

  1. Test-Retest
  2. Equivalent Forms
  3. Test-Retest With Equivalent Forms
  4. Split Half
  5. Kuder-Richardson
• Assessment Methods

  1. Objective Type - Short Answer, completion test
  2. Objective Selection - Multiple choice, True/False, Matching Type
  3. Essay - Restricted, Response Extended
  4. Performance Based - Presentation, Papers, Projects, Athletics, Demonstrations, Exhibitions, Portfolios
  5. Oral Question - Oral Exams, Conferences, Interviews
  6. Observation - Informal, Formal
  7. Self-report - Attitude, Survey, Inventories, Socio-metric Devices
• CLASSIFICATION OF TESTS 

  1. Ability Test 
  2. Aptitude Test
  3. Teacher Made Test
  4. Psychological Test
  5. Survey Test
  6. Mastery Test
  7. Verbal Test
  8. Standardized Test
  9. Power Test
  10. Speed Test
• CHARACTERISTICS OF TESTS WHICH AFFECT RELIABILITY

  1. Length
  2. Spread of Scores
  3. Item difficulty
  4. Item discrimination
  5. Time limits
• Methods of Reliability

  1. Test-Retest is giving a test twice to the same group with any time interval between tests from several minutes to several years.
  2. Equivalent Forms is giving parallel forms of tests with close interval between forms.
  3. Test-Retest with Equivalent Forms is giving parallel forms of tests with increase interval between forms
  4. Split-half is when you give a test once-Score equivalent halves of the test. e.g. odd-even numbered items
  5. Kuder-Richardson happens when you give a test once then correlate the proportion/ percentage of the students passing and not passing a given item.
• SHAPES, DISTRIBUTION AND DISPERSION OF DATA

  1. Symmetricaly shaped test score distributions
  2. normal distribution or bell shaped curve
  3. rectangular distribution
  4. u-shaped curve
  5. Skewed distributions of test scores
  6. positively skewed distribution 
  7. negatively skewed distribution
  8. Unimodal, bimodal and multimodal distributions of test scores
  9. Width and location of score distribution (narrow tail)
  10. Types of kurtotis curves
  11. leptokurtic curve- homogeneous
  12. mesokurtic curve- normal distribution
  13. platykurtic curve –heterogeneous (evenly distributed from highest to lowest)