Internal Validity Threats Practice tests

• Internal Validity


  The degree to which a researcher is able to state a causal relationship between antecedent conditions and the subsequent observed behavior.
• History


  Any event other than the intervention that transpires during the time when the treatment is administered and may affect the observations.
• Maturation


  Some normal developmental changes in participants between the pretest and the posttest that might affect the results
• Testing


  Performance on the post-test may be due to the pre-test, not to the treatment, or, more likely, and interaction of the pre-test and treatment.
• Instrumentation


  Changes in the measuring device or procedure over the course of a study
• Statistical Regression


  The tendency for the extreme scores to regress towards the mean.
• Selection


  Any differences between groups that are due to the differential selection or assignment of subjects to groups.
• Attrition or Subject Mortality


  Any change in overall scores between groups or in a given group over time that may be attributed to the loss of some of the participants.
• Selection

  Interactions

  When selection is coupled with another threat to validity—usually maturation
• contamination


  occurs when participants' knowledge about their assignment to either

  the control group or experimental group may affect their performance
• compensatory rivalry


  efforts by participants in the control group to outperform participants in the treatment group to prove that they are "just as good, if not better."
• resentful demoralization


  participants in the less desirable treatment group (or in a control group) will often become demoralized, which tends to decrease performance.
• Treatment Diffusion


  when treatment delivered to the experimental group is unwittingly spread to the control group
• Compensatory equalization of treatment


  participants in the control group are given compensations that make the comparison groups more equal than originally planned
• A researcher decides to try a new mathematics curriculum in a nearby elementary school and to compare student achievement in math with that of students in another elementary school using the regular curriculum. The researcher is not aware, however, that the students in the "new curriculum" school have computers to use in their classrooms. What threat of internal validity are likely to occur?


  Selection and History (Selection Interaction)-Historyis most likely a threat since the computers in the new curriculum school will interfere with achievement variable in the study.Differential selectionalso could pose a problem since students will not likely be assigned randomly to the schools in which the experiment will occur. Anytime groups in an experiment are not randomly formed, differential selection is likely to pose a problem.
• A researcher wishes to compare two different kinds of textbooks in two high school chemistry classes over a semester. She finds that 20% of one group and 10% of the other group are absent during the administration of unit tests.


  Attrition or Subject Mortality and Selection (Selection Interaction)-Mortalitysince students did not participate in the tests. Also, since intact classes used,differential selection.
• Teachers of an experimental English curriculum as well as teachers of the regular curriculum administer both pre- and posttests to their own students.

  Testing and Selection (Selection Interaction)- A pretest was used, sotestingmay be a possible threat. Also, it is not clear if the groups were randomly formed, so maybedifferential selectionalso.
• Eight-grade students who volunteer to tutor third-graders in reading show greater improvement in their own reading scores than a comparison group that does not participate in tutoring.

  Selection-Differential selection is a problem here since the groups in the study were not selected in the same manner. Maturation is not a problem because there is a comparison group.
• Those students who score in the bottom 10% academically in a school in an economically depressed area are selected for a special program of enrichment. The program includes special games, extra materials, special "snacks," specially colored materials to use, and new books. The students score substantially higher on achievement tests 6 months after the program is instituted.

  Statistical Regression-Regression to the meanis a possible threat since low achieving students only were selected. Also, since there is no comparison group, in this study in gains will be difficult to distinguish from simple maturation effects.
• A researcher designs a study to investigate the effects of simulation games on ethnocentrism. She plans to select two high schools to participate in an experiment. Students in both schools will be given a pretest designed to measure their attitudes toward minority groups. School A will then be given the simulation games during their social studies classes over a three day period while school B sees travel films. Both schools will then be given the same test to see if their attitude toward minority groups has changed. the researcher conducts the study as planned, but a special, unplanned documentary on racial prejudice is shown in school A between the pretest and the posttest.

  Selection, history (Selection interaction) & testing-Differential selectionis possible since intact groups will be used.Historyis a threat since the unplanned documentary will likely impact the dependent variable. A pretest was given, so it is difficult to rule outtestingeffects.
• A researcher uses pre- and posttests of "anxiety level" to compare students given relaxation training with students in a control group. Lower scores in the experimental group result


  Testing- Nothing is clearly a threat in this example, although the pretest could cause testing effects.
• In a experiment of surveying methods, several people failed to return the control group survey.

  Attrition or Subject Mortality-Mortalityis the clear problem with this study.
• Concerned about pretest sensitization, a researcher constructs a test that is extremely difficult, and that is not content valid, and administers it to both the experimental and control groups. The posttest used to measure gains in achievement is not as difficult, and the experimental group shows a slight larger improvement over the control group.


  Instrumentation-Instrumentation is a problem for two reasons. First, the tests do not appear to be content valid, so scores cannot be interpreted accurately. Second, since the two tests, the pretest and posttest, are not of the same difficulty, they lack equivalence forms reliability, which is also an instrumentation threat.
• A researcher uses the same set of problems to measure change over time in student ability to solve mathematics word problems. The first administration is given at the beginning of a unit of instruction; the second administration is given at the end of the unit of instruction, three weeks later. Improvement scores result.

  Maturation-Maturation effectscannot be eliminated as a rival explanation for the improved scores since there is no control group.
• The achievement scores of five elementary schools whose teachers use a cooperative learning approach are compared with those of five schools whose teachers do not use this approach. During the course of the study, the faculty of one of the schools where cooperative learning is not used is engaged in a disruptive conflict with the school principal.

  Select Interaction-History and differential selectionmay be problems. History results from the conflict as this could affect students' performance; differential selection may be a problem since intact groups--schools--were used.
• A researcher tests a group of students enrolled in a special class for "students with artistic potential" every year for six years, beginning when they are aged five. She finds that their drawing ability improves markedly over the years.

  Maturation-Maturationmay be a problem since lack of control group means changes over time could be explained as maturation effects.
• The researcher uses a self-made test to compare the experimental and control group.


  Instrumentation- Only thing that presents itself here is possible instrumentationsince the self-made test may lack validity.
• In an experimental test of alternative forms of the SAT, a group took the traditional SAT test form which lasted approximately four hours, and then took the shortened version which lasted about one hour immediately afterwards.

  Maturation- Maturation is a problem here. The reason is that after taking the SAT for four hours, it is very likely that the participants will be mentally exhausted, so their performance on the shortened version of the SAT may be poorer not due to a weakness of the shortened SAT (which the researchers are testing for), but due to lack of ability to concentrate.