9.2: Dealing With Time-Related Threats and Order Effects

• What is the purpose of controlling time in a research study?

  By controlling the time from one treatment condition to the next, a researcher has some control over time-related threats to internal validity. Shortening the time between the treatments can reduce the risk of time-related threats. This technique can often increase the likelihood that order effects will influence the results.
• Counterbalancing

  In a withion-subjects design, a procedure to minimise threats from order effects and time-related factors by changing the order in which treatment conditions are administered from one participant to another so that the treatment conditions are matched with respect to time.
• What is the goal with counterbalancing?

  The goal is to use every possible order of treatments with an equal number of individuals participating in each sequences. It disrupts any systematic relationship between time and the order of treatment conditions, and thereby eliminates potential confounding from time-related threats or order effects
• What is the relevance of counterbalancing and order effects?

  Counterbalancing is usually discussed in terms of order effects; order effects can change individual scores and means, but when a design is counterbalanced the changes do not influence the mean differences between treatments; thus, the order effects do not threaten the internal validity of the study.
• What is the value of counterbalacning a within-subjects design?

  It prevents any order effects from accumulating in one particular treatment condition; instead, the order effects are spread evenly across all the different conditions so that it is possible to make fair and unbiased comparisons between treatments.
• What are two disadvantages of counterbalancing?

  1. It can distort treatment means; however, this distortion is not important.
  2. Counterbalancing adds the order effects to some of the individuals within each treatment, but not to all of the individuals; thus, the differences between scores are increased within each treatment, which adds variance.
  3. The assumption that order effects are symmetrical is not always justified. It is possible that one treatment can produce more of an order effect than another treatment.
• Complete Counterbalancing

  In within-subjects design, using a separate group of participants for every possible order of treatment conditions. With n = different treatment conditions and n! = different orders.
• What is the idea behind complete counterbalancing?

  The idea behind this is that a particular series of treatment conditions may create its own unique order effect. To completely balance said effects, the research design should use every possible ordering of treatment conditions.
• What is the issue with complete counterbalancing?

  The issue is that the more treatment conditions there are, the more participants are needed for each treatment conditions.
• Partial Counterbalancing

  A system of counterbalancing that ensures each treatment conditions occurs first for one group of participants, second for one group, third for one group, and so on; but does not require that every possible order of treatment conditions be used.
• What differentiates partial counterbalancing to complete counterbalancing?

  Partial counterbalancing uses enough different orderings to ensure that each treatment condition occurs first in the sequence for one group of participants, occurs second for another, third for another and so on; first group receives treatment A first, another group receives A second, another grup receives B third, etc. Each of the other treatments appears once in each ordinal position.