Probability and Significance

• Probability is the statistical likelihood of the results occurring by chance
• There are two main levels of probability that researchers use. 5% is the level that the hypothesis will either be accepted or rejected and if it is accepted, the probability that the results occurred by chance. 1% is used when the researcher needs to be more confident and eliminate as much possibility of the results occurring by chance
• Type 1 of errors is when there is a false positive. This is an incorrect rejection of a null hypothesis. It is an optimistic error where they claim to have found significant results but haven't. This usually occurs because the significance level is too lenient.
• Type 2 of errors is when there is a false negative. This is an incorrect acception of a null hypothesis. It is an pessimistic error where they claim to have found insignificant results but haven't. This usually occurs because the significance level is too stringent.
• When choosing a statistical test, there are two factors which need to be considered; whether it is testing the difference or correlation, and measurement
• There are three levels of measurement in statistical tests:

  • Nominal data: data in categories
  • Ordinal data: data is ordered from highest to lowest
  • Interval data: data is measured in units
• To chose the correct statistical test, you need to determine whether it is testing the difference or correlation. If it is testing the difference, you need to then determine whether the data is related or unrelated. Unrelated data will be collected through independent group design or matches pairs. Related data will be collected through repeated measures. You then need to determine the measurement level and find the correct statistical test.
• Which tests should be chosen for testing the difference?

  Unrelated:

  • Nominal data is tested through chi-squared
  • Ordinal data is tested through Mann-Whitney
  • Interval data is tested through an unrelated T-Test

  Related:

  • Nominal data is tested through a sign test
  • Ordinal data is tested through Willcoxon
  • Interval data is tested through a related T-Test
• Which tests should be chosen for testing correlation?

  • Nominal data is tested through chi-squared
  • Ordinal data is tested through Spearman's Rho
  • Interval data is tested through Pearson's R