Hypothesis Testing

• Hypothesis


  One of the problems we face when conducting research is that when we select samples from populations, we might not get a sample that accurately reflects that population because we do not know the pattern of scores in the underlying population
• Hypothesis testing


  From our sample we have to decide what we think the population is like. Effectively what we do is observe the pattern of scores in the sample and decide which is the most plausible pattern in the population. What our statistical tests do is calculate a probability value, called the p-value. This probability tells us the likelihood of us obtaining our pattern of results due to sampling error if there is no relationship between our variables in the population
• Null hypothesis (H0)


  Currently accepted value for a parameter. States that there is no effect in the underlying population. By effect, it means: A relationship between two or more variables, A difference between two or more different populations, A difference in the responses of one population under two or more different conditions. H1 and H0 are mathematical opposite
• Alternative hypothesis (H1 or Ha)


  Also called research hypothesis. Involves the claim to be tested that claims the difference in results between conditions is due to the independent variable. Can be directional or nondirectional. The research hypothesis is our prediction of how two variables might be related to each other. H1 and H0 are mathematical opposite
• Hypothesis testing


  1. Formulate a hypothesis

  2. Measure the variables involved and examine the relationship between them

  3. Calculate the probability of obtaining such a relationship if there were no relationship in the population (if the null hypothesis were true)
• Logic of null hypothesis testing

  If this calculated probability is small enough, it suggests that the pattern of findings is unlikely to have arisen by chance and so probably reflects a genuine relationship in the population
• Significance level


  Most psychologists and indeed most reputable psychology journals use the convention that a probability of 5% is small enough to be a useful cut-off point. This cut-off probability is often called alpha (α).
• 1. value


  The probability associated with each statistical test. The p-value is the probability of obtaining an observed effect, given that the null hypothesis is true.
• 1. value approach to hypothesis testing

  Uses the calculated probability to determine whether there is evidence to reject the null hypothesis.
• Alpha (α)


  The criterion for statistical significance that we set for our analyses. It is the probability level that we use as a cut-off below which we are happy to assume that our pattern of results is so unlikely as to render our research hypothesis as more plausible than the null hypothesis.
• Significance level


  On the assumption of the null hypothesis being true: if the probability of obtaining an effect due to sampling error is less than 5%, then the findings are said to be 'significant'. If this probability is greater than 5%, then the findings are said to be 'non-significant'.
• Significance level


  The conventional view today is that we should report exact probability levels for our test statistics (the exact p-value or α) and shift away from thinking in terms of whether or not the findings are statistically significant.
• 1. value


  A measure of the probability that an observed difference could have occurred just by random chance. The lower the p-value, the greater the statistical significance of the observed difference. P-value can serve as an alternative to or in addition to preselected confidence levels for hypothesis testing.
• Decision rule (α level)

  Reject H0 if the obtained probability ≤ α, Fail to reject H0 and retain H0 if the obtained probability > α.
• Reporting significance tests


  • Productivity was significantly higher when the workers worked under a bright light environment, p-value < 0.05
  • Productivity was significantly higher when the workers worked under a bright light environment, p-value = 0.023
• Statistical significance is different from psychological significance. Just because a statistically significant difference is found between two samples of scores, it does not mean that it is necessarily a large or psychologically significant difference.
• One of the main problems with the p-value is that it is related to sample size. If, therefore, a study has a large number of participants, it could yield a statistically significant finding with a very small effect (relationship between two variables or difference between two groups).
• Correct interpretation of p-value


  Alpha simply gives an indication of the likelihood of finding such a relationship if the null hypothesis were true. The lower the significance level, the stronger the relationship between two variables. This is not what is meant by the significance of a finding. In fact, we do not know what the probability is that the research hypothesis is correct; our α probability is conditional upon the null hypothesis being true and has nothing to do with the truth or falsity of the research hypothesis. If I set α at the traditional 5% level and find a significant relationship, I can assume that there is a 95% probability that the research hypothesis is true. This is incorrect!
• One-tailed hypothesis


  A one-tailed hypothesis is one where you have specified the direction of the relationship between variables or the difference between two conditions.
• Two-tailed hypothesis


  A two-tailed hypothesis is one where you have predicted that there will be a relationship between variables or a difference between conditions, but you have not predicted the direction of the relationship between the variables or the difference between the conditions.
• Type I error


  The decision to reject the null hypothesis when the Null hypothesis is true. That is, you conclude that there is an effect in the population when no such effect really exists.
• Type II error


  The decision to retain the null hypothesis when the null hypothesis is false. It represents the case when you do not reject the null hypothesis when in fact you should do because in the underlying population the null hypothesis is not true.
• Testing hypotheses


  1. A Type I error, is the rejection of the null hypothesis when it is actually true.

  2. A Type II error, occurs when we fail to reject the null hypothesis when it is actually false.
• Calculating for sample data

  1. Used to decide

  2. Ex: Sample 50 bars - Get average value or mean

  3. Calculate test statistics

  4. Statistically Significant – Where do we draw the line to make a decision?
• Level of confidence (C)

  • 95%
  • 99%
• Alpha (α)


  α = 1 - C. So... if C = 95%, C = 0.95, α = 1 - 0.95, α = 0.05. How confident are we in our decision.
• Evaluating the tail of the distribution

  One-tailed Tests vs Two-tailed Test
• Pagano, R. R. (2020). Understanding Statistics in the Behavioral Sciences (10th ed.). Cengage Learning.