Blog

Posted on

Mr Reliable

Kevin Durant is Mr Reliable

Being reliable is an important and sought after trait in life. Kevin Durant has proven himself to be just that to the NBA. Would you agree (pun intended)? So, we have been talking about agreement lately and we have covered precision (see Repeat After Me) and accuracy (see Men in Tights). Today let’s talk a little about reliability.

 
As I mentioned last time, the concepts of accuracy and precision originated in the physical sciences because direct measurements are possible. Not to be outdone, the social sciences (and later in the Medical Sciences) decided to define their own terms of agreement – validity and reliability.
 
So the concept of reliability was developed to reflect the amount of error, both random and systematic, in any given measurement. For example if you were to want to assess the the measurement error in repeated measurements on the same subject under identical conditions or to measure the consistency of two readings obtained by two different readers on the same subject under identical conditions. 
 
The reliability coefficient is simply the ratio of variability between subjects to the total variability (sum of subject variability and measurement error). A coefficient of 0 indicates no reliability and 1 indicates perfect reliability with no measurement error.
 
Being Mr Reliable (see the trailer to this cool old movie from the sixties) is always desirable but keep in mind that when you consider reliability remember that:
 
1- A true score exists but is not directly measurable (philosophical…)
 
2- A measurement is always the sum of the true score and a random error.
 
3- Any two measurements for the same subject are parallel measurements in that they are assumed to have the same mean and variance.
 
 
With these assumptions in place, reliability can be also expressed as the correlation between any two measurements on the same subject – AKA the intraclass correlation coefficient or ICC (originally defined by Sir Francis Galton and later further developed by Pearson and Fisher). We will talk about the ICC in a later post.
 
Phew! That was a mouthful. All this talk of reliability is exhausting. Maybe Lean on me (or Bill Withers, actually) for a bit and we will talk about validity when we come back…




See you in the blogosphere,






Pascal Tyrrell

Posted on

Men in Tights?

One of the first movies my parents took me to see was Disney’s Robin Hood in 1973. This was back in the days when movies were viewed in theaters and TV was still black and white for most people. One of Robin’s most redeeming qualities is his prowess as an archer. He simply never misses his target. Well maybe not so much in Mel Brook’s rendition of Robin Hood Men in Tights!


We have been talking about agreement lately and last time we covered precision (see Repeat After Me). We discussed that precision is most often associated with random error around the expected measure. So, now you are thinking: how about the possibility of systematic error? You are right. Let’s take Robin Hood as an example. If he were to loose 3 arrows at a target and all of them were to land in the bulls-eye then you would say that he has good precision – all arrows were grouped together – and good accuracy as all arrows landed in the same ring. Accuracy is a measure of “trueness”. The least amount of bias without knowing the true value.  Now if all 3 arrows landed in the same ring but in different areas of the target he would have good accuracy – all 3 arrows receive the same points for being in the same ring – but poor precision as they are not grouped together.



As agreement is a measure of “closeness” between readings, it is not surprising then that it is a broader term that contains both accuracy and precision. You are interested in how much random error is affecting your ability to measure something AND whether or not there also exists a systematic shift in the values of your measure. The first results in an increased level of background noise (variability) and the latter in the shift of the mean of your measures away from the truth. Both important when considering overall agreement.


OK, take a break and watch Shrek Robin Hood. The first of a series is always the best…


Now the concepts of accuracy and precision originated in the physical sciences. Not to be outdone, the social sciences decided to define their own terms of agreement – validity and reliability. We will discuss these next time after you listen to Bryan Adams – Everything I Do from the Robin Hood soundtrack. Great tune.






See you in the blogosphere,




Pascal Tyrrell

Posted on

Michener Institute Series: Princess Margaret Hospital, Toronto, Ontario



As first year Radiation Thearpy students here a The Michener Institute, we are currently in our 4th week of clinical placements! As promised, here’s a little update about the experiences Jennifer and Ori are going through at Princess Margaret Hospital.

Jennifer: I’ve been placed in Unit 10 which specializes in treating patients with Genitourinary, Gynae and Lower Gastrointestinal cancer.

Ori: I’m on Unit 14 and we treat breast cancer and palliative cancers.

We are proud to say that we are enjoying our experience here. Our duty as students in training is to follow the radiation therapist and learn what they do. The job of a therapist is to treat cancer using a machine called Linear Accelerator (Linac) to deliver ionizing radiation. Patients will typically come once a day for the next couple of weeks, so we see the same patients every day and therefore really get to know our patients well. There is a fair amount of patient interaction, which is one of our favorite parts of the job. Along with patient interactions, we also get to use the equipment, which mainly includes operating the Linac machine (the machine that delivers the radiation) and taking X-rays or CT scans to make sure the patient is in the right position. Every day is a new experience and we are constantly learning new skills. We get a better insight of the patient’s perspective during the entire span of their radiation treatment. For example most patients in unit 10 are required to have a full bladder and empty rectum. Having to hold their pee can be quite difficult for some patients, especially when there are delays, which pushes Unit 10 to be a very fast paced environment. Overall our first 4 weeks of clinical has been an exceptionally valuable experience and we’re looking forward to our next 4 weeks!


Until next time!

Jennifer and Ori


Posted on

Michener Institute Series: Clinical Placement Site – Kingston Ontario

 
 
(Kingston City Hall)
It has been a month since the start of summer clinical placement, and I am currently
completing my placement in Kingston General Hospital (KGH) here at Kingston, Ontario.  Kingston is a nice beautiful town located at the north side of the entrance of outflow of St Lawrence River from Lake Ontario; it was the first capital of Canada when Canada was still a province of British colony.
 
KGH host one of the most eastern cancer center in Ontario and it has a beautiful view because it is situated by the side of Lake Ontario, its front entrance open to the water. It is a perfect place for lunch and enjoys the sun during summer time.
 



      (KGH cancer centre front entrance)    
               
          (MacDonald
Park by water, in front of cancer centre)
 



 
The past month was phenomenal, words cannot fully describe the knowledge and experience we gain from clinical practice. The transition from purely academic to hands on
practice is eye-opening and a bit hectic; because each patient is unique and no knowledge from books can prepare you how to interact with all patients.  It is interesting to learn from the therapists, the way they educate patients on their first day of treatment, the type of approach to each patient base on the assessment they do during the conversation with them. It’s amazing how much compassion the therapists have for patients and how much they care for them.
 
 
During the first two weeks in CT simulation unit, I made my first mask and had my own mask made for treatment to head and neck regions. The mask is made of pliable plastics. They come in as a sheet of plastic in a frame, and are put into a warm/hot water bath for 2-4 minutes to makes it pliable, after the mask is taken out of the warm water bath there is a 30-60 seconds window before it hardens. The therapist takes out the mask, tower dry it as much as possible and covers it on patient’s head as fast as possible.  The therapists are very efficient at their job, but what is amazing are the patients going through the process; imagine a warm and moist piece of plastic cover you face, harden in an instant and lock your head into position, and afterword you cannot move for 5-10 minutes for CT scan. I never had thought of the discomfort till I experience it myself.
 
 

 

(My 1st  mask, can kinda see my face print)
 
So far the experience here is amazing, and hopefully the coming June will be equally fantastic as well.
 
Till next time.
 
 

 

Gordon
Posted on

Repeat After Me…

So, in my last post (Agreement Is Difficult) we started to talk about agreement which measures “closeness” between things.  We saw that agreement is broadly defined by accuracy and precision. Today, I would like to talk a little more about the latter.


 The Food and Drug Administration (FDA) defines precision as “the degree of scatter between a series of measurements obtained from multiple sampling of the same homogeneous sample under the prescribed conditions”. This means precision is only comparable under the same conditions and generally comes in two flavors: 


1- Repeatability which measures the purest form of random error – not influenced by any other factors. The closeness of agreement between measures under the exact same conditions, where “same condition” means that nothing has changed other than the times of the measurements.


2- Reproducibility is similar to repeatability but represents the precision of a given method under all possible conditions on identical subjects over a short period of time. So, same test items but in different laboratories with different operators and using different equipment for example.




Now, when considering agreement if one of the readings you are collecting is an accepted reference then you are most probably interested in validity (we will talk about this a future post) which concerns the interpretation of your measurement. On the other hand if all of your readings are drawn from a common population then you are most likely interested in assessing the precision of the readings – including repeatability and reproducibility.


As we have just seen, not all repeats are the same! Think about what it is that you want to report before you set out to study agreement – or you could be destined to do it over again as does Tom Cruise in his latest movie Edge of Tomorrow where is lives, dies, and then repeats until he gets it right… 






See you in the blogosphere,




Pascal Tyrrell

Posted on

Agreement Is Difficult: So What Are We Gonna Do? I Dunno, What You Wanna Do?

It is never easy to come to an agreement – even amongst friends! The Vultures from Disney’s The Jungle Book (oldie but a goodie) certainly know this. In medical research measuring agreement is also a challenge. In this series of posts I am going to talk about agreement and how it is measured.


Agreement measures the “closeness” between things. It is a broad term that contains both “accuracy” and “precision”. So, let’s say you are shopping for screen protectors for your wonderful new phone. You head to the internet and start going through the gazillion links advertising screen protectors of all sizes and styles. As you just spent your savings on the phone, you do not have much money left over for the screen protector. You decide on a generic brand and order a pack of 10. After an unbearable wait of a week to receive them in the mail you open the pack and find that even though you ordered the screen protector to fit your specific phone they are a little small… except for two that fit perfectly! What? That’s annoying.


So, how close are the screen protectors to being “true” to the expected product? This is agreement. Now, most of them are a little small. This represents poor “accuracy”. This is because there exists a systematic bias. If you took the mean size of these 10 protectors you would find that it deviated from the true expected value – size in this case. Furthermore, you found that two of the 10 protectors actually fit your phone screen rather well. This is great, but this inconsistency between your protectors represents poor “precision”. This time we are interested in the degree of scatter between your measurements – a measure of within sample variation due to random error (see my Dickens post for more info).


Now the concepts of accuracy and precision originated in the physical sciences where direct measurements are possible. Not to be outdone, the social sciences (and then soon to adopt medical sciences!) decided to establish similar concepts – validity and reliability. We will discuss these in a latter post but for now simply remember that the main differences are that a reference is required for validity and that both validity and reliability are most often assessed with scaled indices.


Phew! That was a little confusing. Have a listen to We Just Disagree from Dave Mason to relax.


Next post we will look a little more closely at two special kinds of precision – repeatability and reproducibility.




See you in the blogoshere,


Pascal Tyrrell

Posted on

It’s All Relevant According to Einstein… or Was It Relative?

One of Einstein‘s many theories is that a light beam always appears to have the same speed, no matter how fast you are moving relative to it. This theory is also one of the foundations of Einstein’s special theory of relativity. So why the Mini Einstein Bobble Head? Because of the Night at the Museum 2 movie, of course! Have a peek at the trailer and come back.

OK, the last letter in our F.I.N.E.R. mnemonic – a convenient way to remember what makes a good research question – is R. We covered E for Ethical last time and today we will go over R for Relevant – not relative (pay attention now!).

So, you are now a junior researcher with a newly minted pocked protector and have decided to step back a minute and assess your research question using F.I.N.E.R. Among the 5 characteristics we have discussed this last one is an important one. Let’s go back to F is for Feasible where we were thinking of a way to survey your friend’s about going camping at the end of the semester to celebrate the start of summer. The results of your survey will provide you with important information. Not only will they influence your decision to have the event or not, but they will also allow for promotion of the event as being “really popular” (important to many participants) and for better planning (important to the organizing committee). The results of the survey are “relevant”.

 
Make sure the results of your study will contribute to research knowledge and influence change in your field. Maybe a mentor can help shed some light on it if you are unsure. The broader the relevance of your results the better. This touches on “knowledge translation” and we will chat about that later on in our blog.
 
That’s it for FINER! Practice it a little this week-end and let me know how it goes.
 
 
See you in the blogosphere,
 
Pascal Tyrrell 
Posted on

A Little Respect To Avoid The Blues… Brothers?

In her famous song Respect (have a listen while we chat), Aretha Franklin provides us with a perfect segue to the next letter in our F.I.N.E.R. mnemonic – a convenient way to remember what makes a good research question. We covered N for Novel last time and today we will go over E for Ethical.

Everyone has an idea of what they consider to be “right”. Ethics is a branch of philosophy that focuses on studying what is morally right and wrong in our society. Setting standards to live by that will be in everyone’s best interest. More importantly, ethics also involves the continuous effort of studying our own moral beliefs and our moral conduct, and striving to ensure that we live up to these standards that are reasonable and solidly-based.

Now The Blues Brothers may not have operated with completely ethical methods but you should when you perform research. Wondering what the Blues Brothers have to do with any of this? Watch the trailer and you will see Aretha singing “R-E-S-P-E-C-T”. She wants to be treated “right” just as your subjects will in your study.

 
When you think of your research question be sure to consider whether your study will pose unnecessary physical or emotional harm to your subjects – human or otherwise. If you are unsure then you should discuss your question with someone from your institution’s ethics review committee.

But what if you are doing research on your own for fun and are not sure? Ask Mom, she’ll know.

 
Next time we will chat about R for Relevant…
 
 
See you in the blogosphere,
 
 
Pascal Tyrrell

Posted on

New Gold Dream: Is It that Simple?

What a great album from Simple Minds. Ahhh, the 80’s. Their title track New Gold Dream should get you in the mood for the next letter in our F.I.N.E.R. mnemonic – a convenient way to remember what makes a good research question. We covered I for Interesting last time and today we will go over N for Novel. 

 
Your pocket protector in place and armed with an interesting research question that you think is feasible, you are now stuck wondering if the research path you are about to take will satisfy the next criteria: is it novel?

 

The whole idea behind research is to contribute new information. 

 


No need to reinvent the wheel. You want to save your precious energy and time for answering a question that will move you forward in your area of science. 

 
 
 
 
So, how do you know it is novel? Here are a few suggestions you can try:
 
1- Review the scientific literature. And then review it again. Not sure how to get started? Talk to a librarian at your institution.
 
2- Get out there and talk to people about your research idea. Experts in the field will be happy to chat (most of the time anyway!) and may give you insider knowledge about the area of research. How about having lunch with friends/colleagues and ask them what they think. I did exactly that yesterday (shout out to Sindhu Johnson!) and what did I find out? That my ideas were well received and that there is at least one person who thinks I am on the right track. Perfect. Who’s next?
 
 
Now though you want your question to be as original as possible don’t throw out the baby with the bath water. Often building on previous work or simply confirming it can be important. For instance assessing whether findings in one population also applies to others. This is often the case in pediatric research.
 
Having the new gold dream is always a great way to start. Just keep in mind what makes a good research question. Next time we take on ethics…

See you in the blogosphere,

 
 
Pascal Tyrrell
Posted on

Gwen Stefani Has No Doubt… Do You?

So, in my last post I introduced F.I.N.E.R. as a convenient way to remember what makes a good research question. We covered F for feasible and today we will go over I for Interesting. 


What is important when dreaming up a research question is to make sure that you are interested and engaged. This is what will provide you with the energy, drive, and determination to overcome the many hurdles and frustrations that will invariably stand in you way on your path during the research process.


Gwen may have No Doubt about what she is interested in. But do you? How will you gauge how interesting your question is? Easy – talk to people about it. One of the problems new researchers have with their research questions is that they Don’t Speak (great song!) with others during the planning process. Ask as many mentors, experts, family members, friends, colleagues as you can about your question. All that feedback will help you determine whether it is worth your precious time and effort to pursue that research.


Don’t be shy to ask people their opinion and don’t be take it personally if you get negative feedback. It is all part of the process. You can’t expect to have everyone interested but you can certainly try your best to have many. 


Try early on in your research career to find a Person of Interest (well maybe not that kind of person) or someone who you value their opinion and are friendly with to act as a sounding board to your ideas before you move on outside the “inner-circle”. You can even repay the favor to them for their research endeavors. Hint: choose wisely…






Next is N…




See you in the blogosphere,


Pascal Tyrrell