Hypothesis Testing

• Hypothesis


  A statement made about the value of a population parameter
• Hypothesis testing


  1. Carry out an experiment or take a sample from the population

  2. Calculate the test statistic from the sample
• Null hypothesis (H0)


  The hypothesis assumed to be correct
• Alternative hypothesis (H1)


  Tells us about the parameter if the null hypothesis is shown to be wrong
• Example 1: Coin toss

  • John wants to see if a coin is unbiased or biased towards coming down heads
  • He tosses the coin 8 times and counts the number of heads, X, obtained in 8 tosses
• Test statistic


  The statistic calculated from the sample
• The null hypothesis (H0) is rejected if the test statistic is lower than a given threshold, called the significance level
• Critical region


  A region of the probability distribution which, if the test statistic falls within it, would cause you to reject the null hypothesis
• Critical value


  The first value to fall inside of the critical region
• Actual significance level


  The probability of incorrectly rejecting the null hypothesis
• Example 2: Binomial distribution


  • A single observation is taken from the binomial distribution B(6, p)
  • The observation is used to test H0: p = 0.35 against H1: p > 0.35
• Finding critical values


  1. Assume H0 is true

  2. Calculate P(X ≥ 4) and P(X ≥ 5)

  3. The critical region is 5 or 6
• The actual significance level of this test is P(reject null hypothesis) = P(X ≥ 5) = 0.0223 = 2.23%
• Two-tailed test


  • Used to test if the probability is changed in either direction
  • The critical region is split at either end of distribution
  • The significance level at each end is halved
• For two-tailed tests, H1: p ≠ ...
• Example 4: Vegetarian meals

  • In Enrico's restaurant, the ratio of non-vegetarian to vegetarian meals is 2 to 1
  • In Manuel's restaurant in a random sample of 10 people ordering meals, 1 ordered a vegetarian meal
  • Test whether the proportion of people eating vegetarian meals in Manuel's restaurant is different from Enrico's restaurant
• Two-tailed test method


  1. Calculate P(X ≤ 1) and compare to significance level

  2. Find the two critical values c1 and c2 such that P(X ≤ c1) ≤ 0.025 and P(X ≥ c2) ≤ 0.025
• There is no evidence that proportion of vegetarian meals at Manuel's restaurant is different to Enrico's
• One-tailed test


  • Can be used to test if the probability has increased or decreased
  • For one-tailed tests, H1: p > ... or p < ...
• Example 3: New drug

  • The standard treatment for a disease has a 0.4 probability of success
  • A researcher claims a new drug is more effective
  • Test the claim at 5% significance level
• One-tailed test method


  1. Define test statistic X and parameter p

  2. Formulate model X ~ B(20, p)

  3. Identify null and alternative hypotheses

  4. Calculate P(X ≥ 11) assuming H0 is true

  5. Compare probability with significance level
• The new drug is no better than the old one