Statistical Significance and Effect
Size

Is quarter 2 a fair quarter?

Quarter 1 (above): A fair quarter

Quarter 2 (above): A quarter that may or may not be fair.

Is quarter 2 a fair quarter?

Quarter 1 (above): A fair quarter.
50% odds of heads or tails.

After 300 tosses of quarter 2, there were 175 heads and 125 tails.

What is the likelihood of getting the result of heads/tails found with quarter 2 if it is actually a fair quarter?

Did use of RTL products cause greater improvements in scores relative to comparable commercially available products?
How can we be sure?
250 use RTL products every day for a month
500 people are found to participate in a study 250 use comparable commercially available product every day for a month Average test score 9/10

Average test score:
7/10

250 use RTL products every day for a month
500 people are found to participate in a study 250 use comparable commercially available product every day for a month Average test score 9/10

Average test score:
7/10

The question of is this difference significant? = what is the probability of getting this difference in scores if there were no actual difference between the groups?

Significance
• P<0.05
• A result (or difference between two groups) is considered significant if there is less than a 5% of it happening by chance.
• A result that is not considered significant is more than 5% likely to have just happened by chance. What significance is not: Effect Size
A difference can be highly