Week 12/13 Power Point (Research Methods)

• Single Subject Research Design

  A type of quantitative research that involves studying in detail the behaviour of each of a small number of participants
• Single Subject Research Design

  • Quasi-experimental (manipulation of an independent variable, controls for extraneous variables, not random assignment)
  • Time series design (Series of observations before treatment and a series of observation after the treatment)
  • Allows researcher to conduct experiments in which a small number of participants are available (better for applied research)
• Elements of a Single Subject Design

  • Baseline, distinct phases (Case provides its own control)
  • Measured repeatedly across different levels of the independent variable (phases)
  • Phases occur at different intervals (steady state strategy)
  • Replication
• Disadvantages of Single-Subject Designs

  • Relationship among variables is for only one subject
  • May threaten external validity (generalization)
  • Multiple, continuous observations are required
  • Reliance on graphs to demonstrate that treatment effects are real
  • Treatment effects must be large and immediate to produce a convincing graph
• Why use Single-Subject Research

  • Group research can hide individual differences and generate results that do not represent the behaviour of individuals
  • Single-subject research, reveals individual differences
• Why use Single-Subject Design

  • It discovers causal relationships through the manipulation of an independent variable, the careful measurement of a dependent variable, and the control of extraneous variables
  • High internal validity (causal)
  • High social validity
• General Strengths and Weaknesses of Single-Subject Designs

  • Allows researchers to establish see a difference for each subject
  • Flexibility: the researcher is free to modify the treatment or change to a new treatment if a subject fails to respond to treatment
  • No need to standardize treatment across groups—a single subject is used
• Data Analysis


  • Single Subject Designs use visual inspection to determine effects
  • Group researchers worry that visual inspection is inadequate for deciding effectiveness. May not be sensitive enough to detect weak effects and it lacks reliability and cannot be compared across studies
• External Validity


  • Group Researchers may say Difficulty in knowing whether results for just a few participants are likely to generalize to others in the population. Large sample sizes are better likely to generalize to the target population
  • Single Subject Researchers may say A small positive effect in a large group study may indicate many participants exhibited a small positive effect, others exhibited a large positive effect, and still others exhibited a small negative effect. Group researchers study a single situation and then generalize their results to other situations. Multiple data points may control for this. Similar results, from independent researchers support the generality of those results.
• When the behaviour of particular individuals is of interest, Single-Subject Research is effective
• When clinicians work with only one or low number of individuals at a time, Single-Subject Research is effective
• Group Design Research is good for Testing the effectiveness of treatments at group level, Allowing researchers to detect weak effects, which can lead to refinements of the treatment, and Studying interactions between treatments and participant characteristics
• Single Subject Research Design Standards (What works Clearinghouse)


  • Systematic manipulation of independent variable
  • Outcome measures systematically over time (inter-observer agreement)
  • Baseline stability
  • Minimal of 3 attempts to demonstrate an intervention effect (control)
  • Each phase must have a minimal of 3 data points
  • Immediacy of effect (short latency)
  • Examine and control external factors
  • Document consistency of level, trend, and variability within each phase
• Phases and Phase Changes

  • Phase: series of observations of the same individual under the same conditions
  • Baseline observations are made when no treatment is being administered
  • Baseline phase: a series of baseline observations
  • Identified by the letter A
  • Treatment observations are observations made when a treatment is being administered
  • Treatment phase (B): a series of treatment observations
• Visual Inspection


  • Displayed behavioural data
  • Visual analysis
  • Did behaviour change in a meaningful way? If so, to what extent can that change in behaviour be attributed to the independent variable?
  • Variability
  • Level
  • Trend
• Variability


  • Gast and Ledford (2004) considered stability as 80-90% of the data points are within 25% of the median level
  • Different for each phase
  • Median (middle number)
  • 25% or 12.5% above and 12.5 % below median
• Level


  • The Level is a horizontal line drawn through a series of data points within a condition at the point on the vertical axis equaling the average value of the series of measures
  • May describe as high, medium or low
  • The magnitude of the participants' responses
  • Change in average level
  • Immediate change in level
  • Change in trend
• Trends


  • A trend line going up or down implies behaviour will continue following this trajectory
  • Trends can be visually demonstrated with a line of best fit and can also be described in the result section
  • Line of best fit can be created by drawing a straight line with equal data points on either side, Remember must be in same phase
  • Systematic increase or decrease in data path over time
  • Visually estimate direction and degree of slope in each phase
  • Depict with a dashed trend line (line of progress)
  • Increasing, decreasing, zero trend
• Split Middle Line of Progress

  • Divide data points in 2 halves
  • Divide in half
  • Find the median
  • Connect the lines
• Percentage of data points exceeding the median (PEM)

  • According to Tawney and Gast (1984), the lower the percentage of overlap in data points the greater the impact on intervention has on the target behaviour
  • Ma (2006) proposed a calculation of PEM
• Calculating PEM


  • Calculate the median of baseline
  • Extend this across the treatment condition
  • Add the number of data points below if decreasing or above if increasing (10)
  • Divide by the total number of data points and multiply by 100. (10/10 x100 or 100%)
  • PEM line
• Percent Improvement

  • Obtaining the percent of improvement from baseline
  • (Treatment mean)-(Baseline mean) x 100 Baseline mean
  • Can something improve which was never there?
  • Programs which are teaching a new skill, thus an absence of skill, cannot improve.
• Effectiveness of Intervention


  • According to Scruggs and Mastropieri (1998) the effectiveness of the intervention may be classified by the following table
  • PEM Score
  • Effectiveness
  • Over 90%
  • Very effective
  • Between 70 and 90%
  • Effective
  • Between 50 and 69%
  • Questionable
  • Below 50%
  • Not effective
• Inter-Observer Agreement (IOA)


  • Objectively compares independent observations from two or more people of the same events
  • Procedure for enhancing the believability of data
  • Can also rate new observers and observer drift
  • IOA should occur during each phase and researchers should report the percentage of intervals IOA occurred
  • It is convention that the score should be above 80% but there is no set criterion
• How do I get IOA

  IOA is computed by taking the number of agreements between the independent observers and dividing by the total number of agreements plus disagreements. Then multiplied by 100 to compute the percentage (%) of agreement.
• Single Subject Designs


  • Reversal Designs (ABA, ABAB, ABABA)
  • Multiple Baseline
  • Changing Criterion Design
  • Alternating Treatment Design
• Reversal Designs (ABA, ABAB, ABABA)


  • The pattern of behaviour in each treatment phase is clearly different from the pattern in each baseline phase
  • The changes in behaviour from baseline to treatment and from treatment to baseline are the same for each of the phase-change points in the experiment
• Challenges of the Reversal Design

  • It may be unethical to remove it (i.e., self-injury)
  • The dependent variable may not return to baseline when the treatment is removed. (i.e., replacement skill or natural reinforcer)
• Multiple-Baseline Designs

  Allows the researcher to demonstrate that the treatment is responsible for changes in the dependent variable by showing that the dependent variable changes only when the treatment is introduced at different points in time across participants, settings, or behaviors
• Changing Criterion Design


  The criterion for reinforcement is systematically changed within the treatment phase, allowing the researcher to demonstrate a functional relationship between the changing criterion and the dependent variable
• Alternating Treatment Design


  Allows the researcher to compare the effects of two or more treatments by rapidly alternating the treatments within a single session or over a series of sessions
• Advantages of Alternating Treatment Design include flexibility, efficiency, and the ability to directly compare treatments