|Kevin Durant is Mr Reliable
Being reliable is an important and sought after trait in life. Kevin Durant has proven himself to be just that to the NBA. Would you agree (pun intended)? So, we have been talking about agreement lately and we have covered precision (see Repeat After Me) and accuracy (see Men in Tights). Today let’s talk a little about reliability.
As I mentioned last time, the concepts of accuracy and precision originated in the physical sciences because direct measurements are possible. Not to be outdone, the social sciences (and later in the Medical Sciences) decided to define their own terms of agreement – validity and reliability.
So the concept of reliability was developed to reflect the amount of error, both random and systematic, in any given measurement. For example if you were to want to assess the the measurement error in repeated measurements on the same subject under identical conditions or to measure the consistency of two readings obtained by two different readers on the same subject under identical conditions.
The reliability coefficient is simply the ratio of variability between subjects to the total variability (sum of subject variability and measurement error). A coefficient of 0 indicates no reliability and 1 indicates perfect reliability with no measurement error.
Being Mr Reliable (see the trailer to this cool old movie from the sixties) is always desirable but keep in mind that when you consider reliability remember that:
1- A true score exists but is not directly measurable (philosophical…)
2- A measurement is always the sum of the true score and a random error.
3- Any two measurements for the same subject are parallel measurements in that they are assumed to have the same mean and variance.
With these assumptions in place, reliability can be also expressed as the correlation between any two measurements on the same subject – AKA the intraclass correlation coefficient or ICC (originally defined by Sir Francis Galton and later further developed by Pearson and Fisher). We will talk about the ICC in a later post.
Phew! That was a mouthful. All this talk of reliability is exhausting. Maybe Lean on me (or Bill Withers, actually) for a bit and we will talk about validity when we come back…
See you in the blogosphere,
So, in my last post (Agreement Is Difficult) we started to talk about agreement which measures “closeness” between things. We saw that agreement is broadly defined by accuracy and precision. Today, I would like to talk a little more about the latter.
The Food and Drug Administration (FDA) defines precision as “the degree of scatter between a series of measurements obtained from multiple sampling of the same homogeneous sample under the prescribed conditions”. This means precision is only comparable under the same conditions and generally comes in two flavors:
1- Repeatability which measures the purest form of random error – not influenced by any other factors. The closeness of agreement between measures under the exact same conditions, where “same condition” means that nothing has changed other than the times of the measurements.
2- Reproducibility is similar to repeatability but represents the precision of a given method under all possible conditions on identical subjects over a short period of time. So, same test items but in different laboratories with different operators and using different equipment for example.
Now, when considering agreement if one of the readings you are collecting is an accepted reference then you are most probably interested in validity (we will talk about this a future post) which concerns the interpretation of your measurement. On the other hand if all of your readings are drawn from a common population then you are most likely interested in assessing the precision of the readings – including repeatability and reproducibility.
As we have just seen, not all repeats are the same! Think about what it is that you want to report before you set out to study agreement – or you could be destined to do it over again as does Tom Cruise in his latest movie Edge of Tomorrow where is lives, dies, and then repeats until he gets it right…
See you in the blogosphere,
There’s more to the field of medical imaging than a bunch of stuffy radiologists huddled around a couple of monitors. As I mentioned before in my previous post about the history of the imaging technique, the field has undergone a rapid technological advancement in the past century or so, improving the clinical model of visualization. But let’s take a step back from all the scientific stuff for a brief second and look at these developments in a
slightly different light.
During the early stages of medical imaging, X-rays were able to provide people with an initial view of the internal structure of the human body. As limited as that first view may have been, it still played a pivotal role in both challenging and changing people’s perceptions on the human body – to the point where these details would eventually become common knowledge. Without all the major advancements in medical imaging, we could well expect to still be living in the dark.
To really hammer this point home, further advancements in the field would only continue to build on our understanding. What was once the accepted view of the human body has now been given a complete overhaul, thanks to the availability of imaging devices able to produce higher-resolution cross-sectional pictures.
|The SparkNotes illustrated version of this post
So what’s the common thread in all of this? Research, of course. While the idea of research leading to new and exciting developments is a pretty basic concept in and of itself, it’s still an important one to keep in mind. Although the field of medicine is comprised of many different sectors, even at the base level there are plenty of opportunities to contribute meaningful ideas and suggestions. Just because you’re an undergraduate student, that doesn’t stop you from devising an independent thesis in an area you’re passionate about. Granted, I don’t want to be too idealistic here, given the logistics of funding, but an interesting and relevant pitch to your primary investigator
could go a long way. Who knows, you may find yourself presenting your findings at a research symposium, complete with nifty results and statistics to showcase your efforts.
The bottom line is, a little can go a long way, and if you already have a keen interest in science to start contributing as soon as possible. The entire medical field is driven by people with a knack for research and discovery – and while there’s never a shortage of great minds, there’s always room for more.
Thanks for reading,
Oddly enough, there’s been a surprising lack of content about medical imaging on a blog with medical imaging in its title. So in order to fill that void, I’ll be providing a brief history on the development of the clinical technique used to visualize the human body.
The advent of medical imaging dates all the way back to 1895, following the discovery of X-rays by the German physicist, Wilhelm Conrad Roentgen. The first X-ray picture was then produced, detailing the skeletal composition of his wife’s left hand. However, the actual quality of this imaging process was still very primitive, only allowing for the visualization of bones or foreign objects.
|Much to Dr. Roentgen’s pleasure, Mrs. Roentgen
had not discarded her wedding ring
It was not until the 1920’s that radiologists would develop a more effective method of visualization. This process, known as fluoroscopy, involved either an oral or vascular injection of a radio-opaque contrast agent, which would travel through the patient’s gastrointestinal track. Radiologists could then take films tracking the agent, allowing them to view blood vessels and digestive tracks alike.
By the 1950’s, imaging procedures progressed towards nuclear medicine, involving radioactive compounds. These compounds were administered to patients because they could be absorbed by cellular clusters being invaded by tumours. As compounds decayed and emitted gamma rays, the recorded radiation could then be detected by gamma cameras, signalling the location of any cancerous developments.
The 1970’s were a period of rapid advancement for the field, as a number of modern imaging techniques were developed for clinical practice such as:
- Ultrasound – Uses sound waves that are able to penetrate cellular tissue. Once they reflect off the body’s internal organs, the vibrations generate an electrical pulse which can then be reconstructed into an image.
- PET-CT Scan – Positron emission tomography (PET) uses compounds that emit positrons when they decay rather than gamma rays. It is now combined with a computed tomography (CT) device to generate a high-resolution image displaying sectioned layers of the scanned area.
- MRI – A Magnetic Resonance Imaging scanner runs a strong magnetic field through the body, aligning hydrogen protons. As the protons return to their original position in the atom, they generate radio waves, which are then picked up by the scanner and used to create an image based on signal strength.
Fast-forward to present day and over 70 million CT scans, 30 million MRI scans and 2 billion X-rays have been performed worldwide! The field of medical imaging is still growing by the day, with ongoing research leading to new developments.
Thanks for reading,
Why do we need to find out things, shouldn’t we be content with what we have already, why does research matter? Well, simply put, we conduct research because we are eager human beings looking to seek further knowledge into any task or question presented to us. Human beings in general are inquisitive beings that become fulfilled when they accomplish something and the information spectrum is broadened. To accomplish almost any goal, whether it be recreational or academic, information through research is required. Research essentially helps humans, be human. We question, research and come up with an answer, and that in itself is a true accomplishment.
Research, like any other entity, has its ups and downs. You can follow the processes, look in the right books, but come up with the wrong information. When that happens, do not give up your search, rather organize it another way. Even if that seems to be time wasted, it was not, because the amazing thing about seeking knowledge is even if it’s the wrong knowledge found, one still learns from ANY obtained information. Research is a definite experience, and something valuable is ALWAYS learned when conducting research, whether it be the information being searched for, or a bit of self growth. And the best part? There is no time limit to knowledge. So get started. Just remember your keywords.
Check out MiVIP’s official twitter account! @MiVIP_UofT
Ok, so you agree to dress your little friend before sending him/her out into the cold world of publication. But what is a p-value anyway? I realize that I am jumping the gun (pun intended) a little as it forces us to talk about inferential statistics – a challenging topic. So today I will only give you a small taste of what is to come. First, to get you in a good mood I want you to watch the trailer for the first of three hilarious Naked Gun movies.
We have already talked about research questions and today I would like to introduce you to their children the research hypotheses. Essentially they are a version of their parents that summarize the main elements of a study – sample, predictor and outcome variables – in such a way that you are able to perform a test of statistical significance. These hypotheses are not required for descriptive studies like the ones we have been discussing in our blog so far. For instance if we were to ask how many people who read this blog enjoyed the Naked Gun series of movies we would end up with a proportion. We could then simply describe our findings as discussed in my Ogive post.
But what if you wanted to now if the proportion of gals differed from the proportion of guys who enjoyed the movies as you suspect that the type of humor will please guys more than gals? As we are research scientists we would want to test this “hypothesis” in order to compare the findings among the groups: this is a test of statistical significance. The brilliant statistician Ronald Fisher championed this approach. Only a single hypothesis is required: the null hypothesis. It simply states that no association of interest exists. So in this case whether you are a gal or a guy is not associated with whether you like the Naked Gun movies or not in the population of blog readers.
Break! Listen to the music of P-value Diddy (he has so many names already I thought it ok to add one more) with Jimmy page from the Godzilla soundtrack.
Welcome back. So the null hypothesis is always assumed to be true until shown to be false with a statistical test. When you analyze your data and perform the test you will determine the probability of seeing an effect as big or bigger than that in your study by chance alone if the null hypothesis were true. You would reject the null hypothesis if the p-value is less than a predetermined level of significance – typically 5% or 1 in 20.
So what is a naked p-value? It is simply a p-value obtained from the statistical test you performed on the data from your study reported WITHOUT an effect size, its sign and precision. The effect size is simply an estimate of the size of the association that you are studying – 25% more guys liked the movies as compared to the gals. The sign and precision is simply the direction of the observed difference (are you comparing gals to guys or the other way around) and an estimate of how confident you are – generally reported as a confidence interval which we will talk about in a later post.
So what is the bottom line? In order to keep your p-value warm you need to report it with the measure of the size of the association (effect size) and how confident you are about your answer.
In a subsequent post we will talk about another similar approach, Pearson-Neyman hypothesis testing, which involves two competing hypotheses (the null and the alternate hypotheses). This approach is duductive as opposed to Fisher’s inductive statistical testing approach. Both approaches are valid. It is simply a matter of determining which is more appropriate in a given situation.
See you in the blogosphere,
So now that you are a debutante research scientist you are eager to take your newly acquired skills for a test drive. But where? You could get a job as a research assistant, but jobs are scarce. Apply for a student research summer scholarship. Maybe, but how about that summer job as a lifeguard at Camp North Star (from the crazy movie Meatballs) that you have committed to?
How about volunteering? But why would you do that? Let me tell you why. First listen to CeeLo Green to get pumped (yes, it is about firefighters but just pretend he is signing about volunteer scientists…).
There are many, many, many reasons to be a volunteer for any organization in your community. I am certain you can think of a bunch within a short brain storming session. What I want to share with you is some of my reasons. I have volunteered with special needs in my community for most of my adult life (quick calculation puts me at about 5,000 hrs to date). People’s first reaction is that of surprise. What? How do you have the time? Then they think how admirable… Sure, I’ll take that. But really, I do it because it gives me opportunity. Opportunity to learn and grow. Not only am I happy to do it and it makes others happy doing it, but I accomplish something that I may not have had the opportunity to do otherwise. Think about that. Make others happy AND gain some experience in the process.
Just like in the fantastic movie Shrek Donkey volunteers his services. He is always enthusiastic and willing to help. Listen to him ask to be picked here: Pick me! What does it get him in the end? A couple of great friends and a dragon wife. Perfect.
Create opportunities for yourself by volunteering. You will be glad you did.
Happy Valentine’s Day,
In “A Few Good Men” Jack Nicholson growls “You can’t handle the truth” to Tom Cruise in his Academy award winning performance. Watch a clip of his gritty performance: A few good men. Our pursuit of the truth leads to an interesting path indeed.
This series of posts has as objective to help you develop a scientific “sense”. Have a quick peek at my other posts (http://mivip-utoronto.blogspot.ca/) if you haven’t already and come back. So wanting to know the truth is something we all strive for on a daily basis. Finding the truth is another matter altogether and this philosophical conundrum has challenged many great minds for centuries.
The Roman Emperor Marcus Aurelius once stated many, many years ago: “Everything we hear is an opinion, not a fact. Everything we see is a perspective, not the truth”. Have a quick peek at the trailer for “Gladiator” to put you in the mood. Gladiator
Now Greek philosopher Plato, who predated Marcus a few centuries, got the ball rolling when he presented his Allegory of the Cave, in which he symbolically described his belief that the world revealed by our senses is not the real world but only a poor copy of it, and that the real world can only be apprehended intellectually. Plato used an analogy where we are represented as a gathering of people who live chained to the wall of a cave all of our lives, facing a blank wall. We watch shadows projected on the wall by things passing in front of a fire behind them, and begin to designate names to these shadows. The shadows are as close as we get to viewing reality.
Getting too serious? Take a break and listen to Siouxsie And The Banshees – Shadowtime Shadowtime
So why all the philosophy? Because the concept of getting as close to the truth as possible is important. We accept that the truth will never be known and, therefore, we must also accept as an answer an estimate (let’s say the mean of a sample) or “best guess”. As a scientist we will make sure to offer our reasoning and methodology as to how we obtained this estimate and more importantly we will offer a measure of how confident we are about this estimate – voila, biostatistics in a nutshell. Don’t believe me? Keep reading my posts and I will explain.
If we always knew the truth, would we need to measure anything? How boring would that be? As William Cowper aptly put it: “Variety’s the very spice of life, That gives it all its flavor”.
Here is what I suggest you do next in your endeavor to become a researcher: keep on asking crazy numbers of questions but now think of what factors will influence the estimate you will produce for your answer. Where does this “variety” that Cowper mentions come into play?
Next we will talk about the concept of “expectation” and how this is important in the world of scientific research.
How is that pocket protector working for you so far?
How in the world does someone actually get to researching something, it seems too complicated! Well, fear not my friends, research comes in all letter shapes and information sizes! For a beginner researcher, an easy and effective start would be to go to a library. The library holds a haven of information, from fictional stories for experience to non fictional magazines, dictionaries and novels. The library is a great place to start looking for answers, the librarians are able to guide anyone to the right factual source of data that will provide good information, and no, they aren’t from the dinosaur era (even though you may think you’re in that era while searching through books)..
Many educational researchers use books and even textbooks to help with research projects, papers and various journals. I know I can always trust my school librarian, she always gave me the best books and information, even if she didn’t remember my name.
Even though libraries are ‘so 90s,’ it wouldn’t kill you to walk into one, you definitely will learn something.
Follow us on twitter for the latest updates of the program! @MiVIP_UofT
So you now spend a minimum of an hour a week wearing your shirt with a pocket protector thinking, among other things, about what you can do to speed up your training to become a scientist. Don’t know what I am talking about? Go and see my previous post and come back.(Pocket protector)
Ok. You are now asking questions furiously at all times of the day (and night?) trying to get a handle on how to structure a question in order to best help with finding an answer. Why? It’s all about clarity. Not sure what that is? Listen to Zedd for some instruction: Clarity – Zedd
A great French author Marcel Proust – yes another French author, my first name is Pascal after all – said: “The voyage of discovery lies not in seeking new horizons, but in seeing
with new eyes.” Maybe by asking the right questions we can inch ever so slowly towards the truth that lies right in front of our own eyes! So take a fresh look at what and how you do all things scientific.
Here is what I suggest for formulating your questions:
Use the PICO model (for a little more detail: PICO)
• Patient, Population, Problem
• Comparison (optional. PIO when absent!)
Essentially in a clinical setting – For a patient with (Problem), how does (Intervention) compare to (Comparison) with regard to (Outcome)?
- Is MR angiography more effective than a Doppler carotid ultrasound in diagnosing and describing carotid artery disease in obese middle-aged males and females?
or PIO – For a patient with (Problem), does (Intervention) affect (Outcome)?
- Is a MR angiography effective in diagnosing and describing carotid artery disease in obese middle-aged males and females?
PICO can be applied to most research questions that you may have – yes even outside of Medical Imaging and in the real world (see Scientific thinking in business).
Just remember that you will most probably want to formulate and test a hypothesis based on your research question. For quantitative statistical analysis you will want your question to be answerable by yes/no or a number. For qualitative analysis your question will typically start with: What is/are…?
Keep practicing and we will chat about testing hypotheses next post. Stay tuned…