If Only I Had A Brain… – Tyrrell4innovation

So how was March break? My family and I went to Stowe, Vermont for a little skiing. Awesome. However, the 8 hour drive with 3 kids, our luggage, skis, snowboards, and snacks to get there… maybe not so much. We felt a little like Dorothy in the The Wizard of Oz.

Last time we were talking about p-values and inferential statistics (see Naked p-value if you don’t remember) and I mentioned that I would talk a little more about hypothesis testing. Now Ronald Fisher believed that if you obtained a large p-value when performing a statistical test then you would reject the null hypothesis. So the null hypothesis is always assumed to be true until shown to be false with a statistical test. This helps you determine the probability of seeing an effect as big or bigger than that in your study by chance alone if the null hypothesis were true. This is called significance testing.

Now two other great statisticians – Jerzy Neyman and Egon Pearson – were concerned about the possibility of rejecting a hypothesis that was obviously true. What if the statistical test at hand was NOT being applied correctly? So basically, it would be unreasonable to test whether your data is a certain way (significance testing) unless you assumed that there was other possibilities for your data. This became what is known as the alternative hypothesis. Interestingly, the probability of detecting that alternative hypothesis, if it is true, is called the power of the test. We will talk more about power later in the blog.

So the power of a statistical test is a measure of how good the test is. The more powerful of two tests is the better one to use.

Here is an interesting thought: in many (most?) situations the statistical test you perform for the your study is to test the null hypothesis that no difference in effect exists between groups. In our previous example we were interested in whether gals or guys are associated with whether they like the Naked Gun movies or not in the population of blog readers. If no difference truly existed between gals and guys then why perform the study? The null hypothesis that both gals and guys equally like the Naked gun movies is a “straw man” meant to be knocked down by the results of your study. Therefore, you should always maximize the power of your study in order to knock down the straw man and show a difference exists between gals and guys.

Ok. Now that we have worked up a sweat knocking down Scarecrow from the Wizard of Oz, cool down listening to Long December (yes, I am happy spring is around the corner) from the Counting Crows and…

I’ll see you next time in the blogosphere.

Pascal Tyrrell