It’s Cold Out Today – Please Remember to Dress Your Naked P-Value…

Ok, so you agree to dress your little friend before sending him/her out into the cold world of publication. But what is a p-value anyway? I realize that I am jumping the gun (pun intended) a little as it forces us to talk about inferential statistics – a challenging topic. So today I will only give you a small taste of what is to come. First, to get you in a good mood I want you to watch the trailer for the first of three hilarious Naked Gun movies.

We have already talked about research questions and today I would like to introduce you to their children the research hypotheses. Essentially they are a version of their parents that summarize the main elements of a study – sample, predictor and outcome variables – in such a way that you are able to perform a test of statistical significance. These hypotheses are not required for descriptive studies like the ones we have been discussing in our blog so far. For instance if we were to ask how many people who read this blog enjoyed the Naked Gun series of movies we would end up with a proportion. We could then simply describe our findings as discussed in my Ogive post. 

But what if you wanted to now if the proportion of gals differed from the proportion of guys who enjoyed the movies as you suspect that the type of humor will please guys more than gals? As we are research scientists we would want to test this “hypothesis” in order to compare the findings among the groups: this is a test of statistical significance. The brilliant statistician Ronald Fisher championed this approach. Only a single hypothesis is required: the null hypothesis. It simply states that no association of interest exists. So in this case whether you are a gal or a guy is not associated with whether you like the Naked Gun movies or not in the population of blog readers.

Break! Listen to the music of P-value Diddy (he has so many names already I thought it ok to add one more) with Jimmy page from the Godzilla soundtrack.

Welcome back. So the null hypothesis is always assumed to be true until shown to be false with a statistical test. When you analyze your data and perform the test you will determine the probability of seeing an effect as big or bigger than that in your study by chance alone if the null hypothesis were true. You would reject the null hypothesis  if the p-value is less than a predetermined level of significance – typically 5% or 1 in 20.

So what is a naked p-value? It is simply a p-value obtained from the statistical test you performed on the data from your study reported WITHOUT an effect size, its sign and precision. The effect size is simply an estimate of the size of the association that you are studying – 25% more guys liked the movies as compared to the gals. The sign and precision is simply the direction of the observed difference (are you comparing gals to guys or the other way around) and an estimate of how confident you are – generally reported as a confidence interval which we will talk about in a later post.

So what is the bottom line? In order to keep your p-value warm you need to report it with the measure of the size of the association (effect size) and how confident you are about your answer.

In a subsequent post we will talk about another similar approach, Pearson-Neyman hypothesis testing, which involves two competing hypotheses (the null and the alternate hypotheses). This approach is duductive as opposed to Fisher’s inductive statistical testing approach. Both approaches are valid. It is simply a matter of determining which is more appropriate in a given situation.

See you in the blogosphere,

Pascal Tyrrell