Category: Research Opportunity Program

Posted on

Adam Adli’s ROP399 Journey in Machine Learning and Medical Imaging

My name is Adam Adli and I am finishing the third year of my undergraduate studies at the University of Toronto specializing in Computer Science. I’m going to start this blog post by talking a little bit about myself. I am a software engineer, an amateur musician, and beyond all, someone who loves to solve problems and treats every creation as art. I have a rather tangled background; I entered university as a life science student, but I have been a programmer since my pre-teen years. Somewhere along the way, I realized that I would flourish most in my computer science courses and so I switched programs in at the beginning of my third year.
 
While entering this new and uncertain phase in my life and career, I had the opportunity of meeting Dr. Pascal Tyrrell and gaining admission to his research opportunity program (ROP399) course that focused on the application of Machine Learning to Medical Imaging under the Data Science unit of the Department of Medical Imaging.
 
Working in Dr. Tyrrell’s lab was one of the most unique experiences I have had thus far in university, allowing me to bridge both my interest in medicine and computer science in order to gain valuable research experience. When I first began my journey, despite having a strong practical background in software development I had absolutely no previous exposure to machine learning nor high-performance computing.
 
As expected, beginning a research project in a field that you have no experience in is frankly not easy. I spent the first few months of the course trying to learn as much about machine learning algorithms and convolutional neural networks as I could; it was like learning to swim in an ocean. Thankfully, I had the support and guidance of my colleagues in the lab and my professor Dr. Tyrrell throughout the way. With their help, I pushed my boundaries and learned the core concepts of machine learning models and their development with solutions to real-world problems in mind. I finally had a thesis for my research.
 
My research thesis was to experimentally show a relationship that was expected in theory: smaller training sets tend to result in over-fitting of a model and regularization helps prevent over-fitting so regularization should be more beneficial for models trained on smaller training sets in comparison to those trained on larger ones. Through late nights of coding and experimentation, I used many repeated long-running computations on a binary classification model for dental x-ray images in order to show that employing L2 regularization is more beneficial for models training on smaller training samples than models training on larger training samples. This is an important finding as often times in the field of medical imaging, it may be difficult to come across large datasets—either due to the bureaucratic processes or financial costs of developing them.
 
I managed to show that in real-world applications, there is an important trade-off between two resources: computation time and training data. L2 regularization requires hyperparameter tuning which may require repeated model training which may often be very computationally expensive—especially in complex convolutional neural networks trained on large amounts of data. So, due to the diminishing returns of regularization and the increased computational
costs of its employment, I showed that L2 regularization is a feasible procedure to help prevent over-fitting and improve testing accuracy when developing a machine learning model with limited training data.
 
Due to the long-running nature of the experiment, I tackled my research project as not only a machine learning project but also a high-performance computing project as well. I so happened to be taking some systems courses like CSC367: Parallel Programming and CSC369: Operating Systems at the same time as my ROP399, which allowed me to better appreciate the underlying technical considerations in the development of my experimental
machine learning model. I harnessed powerful technologies like Intel AVX2 vectorization instruction set for things like image pre-processing on the CPU and the Nvidia CUDA runtime environment through PyTorch to accelerate tensor operations using multiple GPUs. Overall, the final run of my experiment took about 25 hours to run even with all the high-level optimizations I considered—even on an insane lab machine with an Intel i7-8700 CPU and an Nvidia GeForce GTX Titan X!
 
Overall, my ROP not only opened a door to the world of machine learning and high-performance computing for me but in doing so, it taught me so much more. It strengthened my independent learning, project management, and software development skills. It taught me more about myself. I feel that I never experienced so much growth as an academic, problem-solver, and software engineer in such a condensed period of time.
 
I am proud of all the skills I’ve gained in Dr. Tyrrell’s lab and I am extremely thankful for having received the privilege of working in his lab. He is one of the most supportive professors I have had the pleasure of meeting.
 
Now that I have completed my third year of school, I’m off to begin my year-long software engineering internship at Intel and continue my journey.
 
Signing out,

Adam
Adli

Posted on

Step 1 in ROP399 – What’s my project?

This week I finally decided on my project topic!

During last week’s lab meeting, Dr. Tyrrell brought up some potential topics for us to choose from. This included determining the appropriate sample size for machine learning, class imbalance problem, participating in the dental project and the ultrasound project that has just been brought up.

After the lab meeting, I talked to Wenda and Ariana regarding the dental project that they have been working on. This was the project that I wanted to be in the most primarily because I intend to go to dental school after graduation, and being involved in a dental project would offer more exposure to this field. However, after the brief introduction and update on the current progress by Wenda and Ariana, I realized that there might not be much to do as a complete project. Hanatu, an independent research student, would also be working on this project, leaving fewer gaps that need to be addressed for the project. Because my expectation is to work on a project independently on a topic where there’s plenty of freedom, I decided to change gears and look at other ideas.

The class imbalance topic was the next thing that caught my interest. Indranil, who happened to be my mentor before I joined the lab, has been working on the class imbalance project before. I immediately contacted him regarding this project and got his project report. I was told that this topic is more technical and less clinical than the dental project, so I didn’t know if I would like the topic. Surprisingly, I found it really interesting and has great implications. Indranil studied the effect of class imbalance using images in the IRMA database and applied the random forest model. By manually changing the sample size of one class, he found that as the proportion of the imbalanced set goes up, the overall accuracy of the model decreases, while the accuracy for the imbalanced class increases. I found it interesting and useful, as class imbalance can be very common in any dataset, especially in medical imaging. Studying its effect can help identify this issue when machine learning is applied to assist with medical imaging.

I then met with Indranil on the possible projects on this topic, and the most natural one would just be investigating which method can better mitigate the class imbalance problem – as a continuation after studying its effects. Next, I researched on any existing literature on this topic specifically in medical imaging, and very little was found. The most commonly used methods for class imbalance include over-sampling, under-sampling, and changing the weight for the imbalanced class coefficient in the cost function. I met with Dr. Tyrrell, he liked the idea for my project, and suggested that I focus on these 3 main methods (mentioned above).

I am excited about my project (and most importantly, really interested). I decide to ask for the code that Indranil used to do the image preprocessing and creating imbalanced classes as a starting point. For my next steps, I’m also planning to learn more about the different methods in addressing this problem as well as how to code in Python.

Looking forward to working on my project!

Wendi
Sep.28, 2018

Posted on

Summer 2018 ROP: Wenda’s in the house!

Hello everyone, my name is Wenda Zhao. I’m starting my fourth year in September majoring in neuroscience and pathobiology. I did a research opportunity project (ROP) 399 course with Dr. Tyrrell this summer. And I’m here to share some of my experiences with you.
Today is a hot and humid Friday in southeast China, where I’m back home from school for the rare luxury of a short break before everything gets busy again. Summer is coming to an end, so is my time with Dr. Tyrrell and his incredible team, some of whom I have got to know, spent most of the summer working with and befriend. I have just handed in my report for the project I did over the past three months on the segmentation, characterization and superimposition of dental
X-ray artifacts. And now, looking back, it was one of the best learning experiences I have ever had, through an enormous amount of self-teaching, practicing, troubleshooting, discussing and debating. As with all learning experiences, the process can be long and bewildering, sometimes even tedious; yet rewarding in the end.
 
It all began on a cold April morning, with me sitting nervously in Dr. Tyrrell’s
office, waiting for him to print out my ROP application and start off the interview. At that point, I just ended my one-year research at a plant lab and was clueless of what I was going to do for the following summer. Coming from a life science background, I went into this interview for a machine learning project in medical imaging knowing that I wasn’t the most competitive candidate nor the most suitable person to do the job. Although I tried presenting myself as someone who had had some experience dealing with statistics by showing Dr. Tyrrell some clumsy work I did for my previous lab, the flaws were immediately noticed by him. I then found myself facing a series of questions which I had no answers to and the interview quickly turned into what I thought to be a disaster for me. I was therefore very shocked when I received an email a week later from Dr. Tyrrell informing me that I had been accepted. I happily went onboard, but joys aside, part of me also had this big uncertainty and doubt that later followed me even to my first few weeks at the lab.
 
At the beginning, everything was new. I started off learning the software KNIME, an open-source data analytics platform that is capable of doing myriads of machine learning tasks. I had my first taste doing a classification problem, where we trained a decision tree model to identify a given X-ray to either be of a hand or a chest. It was a good introductory task to illustrate all the basic concepts in machine learning such as “training set”, “test set”, “input” and “output/label”. We ended up obtaining an accuracy of around 90% on the test set. That was the first time I witnessed the power of machine learning and I was totally amazed by it. I spent the next week or so watching more videos on the topic including state of the art algorithms such as convolutional neural network (CNN). While absorbing knowledge everyday was fun, I was at the same time a little lost about the future of my project. I began to realize that this experience is going to be very different from my past ones in wet labs, where a lot of the times you were already told what to do and all you need is to conduct the experiments and get the results. Here the amount of freedom that I have on my schedule, task and even the project itself was refreshing but at the same time terrifying. On retrospect, I considered myself lucky for that it was around that time of lost when the Faculty of Dentistry proposed a collaboration with us, which ended up being my project for the summer.
 
The dentistry project, as we so called, concerns a type of dental X-ray sensor called Phosphor Storage Plates (PSPs) which are very commonly used because of its easy placement in the oral cavity and the resulting minimum discomfort. The sensors, however, can accumulate damages over time, which would show up in the final image as artifacts with various appearances. Such artifacts could get in the way of diagnosis; thus, the plates need to be discarded before it’s too damaged. But how damaged is too damaged? For the moment, nobody has answers to that. Our goal is to use machine learning to learn the relationship between artifacts and whether they would affect diagnosis. Eventually, we can use that model to make predictions for a given plate and offer dentists advice as in when to discard it. The entire project is huge and the part we played in this summer mainly contributes as preparatory work. We segmented the artifacts from the image and clustered them into five groups based on 9 hand-engineered features. This characterization of the single artifacts can serve as the input for the model. We also created a library of superimposed images of artifact masks and real teeth backgrounds to mimic images taken with damaged sensors in real clinical settings. We did this so that dentists can take a look at these images and give a diagnosis. Comparing that with the true diagnosis, we can obtain the labels for whether a given artifact will affect diagnosis or not. And this will be the output of the model. The testing of these images is currently underway, and the results will be available in early September for further analysis.
 
With the project established and concrete goals ahead, the feeling of uncertainty
gradually went away. But it was never going to be easy. There were times when
we hit the bottleneck; when our attempts have failed miserably; when we had to give up on a brilliant idea because it didn’t go our ways. But after stumbling through all the challenges and pitfalls, we found ourselves new. I was a bit lost at the beginning of this summer. But over the summer I learned a lot about the very cool and growingly crucial field of machine learning; I grew a newfound appreciation for statistics and methodology; I picked up the programming language python, which I had been wanting to do for years and, most importantly, I did more thinking than I ever would if I were to just follow instructions blindly. And in the end, I believe that science is all about thinking. So for you guys out there reading the blog, if you’re coming to this lab from a totally different background and not entirely sure about the future, don’t be afraid. And I hope you find what you come here looking for, just like I did.
 
Finally, I want to thank the people who’s helped me along the way and who’s made the lab such an enjoyable place: Hershel, Henry, Rashmi, John and Trevor; and last but not least, Dr. Tyrrell, without whose kindly offer and guidance I would never have had such an amazing experience. Here’s to an unforgettable summer and a strong start of the new school year. Cheers!
 
Wenda Zhao
Posted on

ROP299 2017-2018: A Medical Imaging Journey from a Humanities Perspective

My name is Samantha Santoro, and I am completing my second year in the English and Biology majors at the University of Toronto, St. George. A rather unconventional combination, when reviewing past students of Dr. Tyrrell’s lab. I was a 2017-2018 Research Opportunity Program (ROP) student in Dr. Pascal Tyrrell’s lab, and my work chiefly consisted of evaluating the internal vessel wall volumes of carotid arteries in a particular cohort of patients provided by the ongoing prospective CAIN study. My ROP was in the field of Medical Imaging. I am the co-president of the student club known as Watsi, with the main chapter based in San Francisco. I am also a special contributor to the Rare Disease Review, along with volunteering at an amalgamation of charity walks and fundraisers.

My ROP project was a turbulent experience – although that word is typically associated with a negative connotation, I regard my ROP299Y1 as one of the most humbling, interesting, and educative experiences that I have had thus far – most definitely not negative. However, to say everything went smoothly would be discrediting the lessons I learned from when things were not idyllic and smooth. My project, as aforementioned, statistically analyzed data provided by patients part of the CAIN study (an analysis that could not have existed without Dr. Tyrrell’s generous and unwavering support). My study determined that patients who were found to have IPH, or what is known as intraplaque hemorrhage, when I analyzed their MRIs, were also found to have increased vessel wall volume. This conclusion is incredibly significant, as IPH is a surrogate marker for atherosclerosis and could potentially be an indicator for patients at risk of future cerebrovascular events (namely, ischemic stroke). As strokes are currently the number three killer in the U.S and Canada alone, and heart disease number one, having a potential indicator for patients at risk of stroke would greatly benefit clinicians in their practice, as well as patients themselves.

As aforementioned, studies similar to my own are currently underway by the Canadian Atherosclerosis Imaging Network, furthering the important research in this field. The VBIRG (Vascular Biology Imaging Research Group) was the lab in which I primarily worked throughout the course of my ROP, at Sunnybrook Hospital. Moreover, I also worked on systematic reviews and reports outside of the focus of my project, in the fields of medical ethics and AI in the radiology workplace – both of which were opportunities provided to me by Dr. Tyrrell, and both of which were incredibly valuable experiences, allowing for me to broaden my knowledge of certain areas of medicine and science that are developing and expanding.

Although my project was littered with its own respective difficulties – a substantial number of drafts throughout each step of the program (more than I had ever made, even being an English student); a reluctant, but later fulfilling, acquaintanceship with the post-processing software VesselMass; and several late nights learning about the field of statistics – it is in light of these difficulties, and at present having overcome them throughout my ROP, that I remember Dr. Paul Kalanithi’s words in his memoir When Breath Becomes Air: “It occurred to me that my relationship with statistics changed as soon as I became one”. He, too, had studied Biology and English. I may not have played a lead role in the statistics I had been working with, but I can now say that understanding what they meant and how they were formulated has generated a deep respect in me for the field of statistics.

My poster was on display at the 2018 Research Opportunity Undergraduate Fair. Special thanks to Mariam Afshin, my supervisor at Sunnybrook Hospital; Bowen Zhang, for answering each question I had while at Sunnybrook; John, and the rest of the lab team; and Dr. Pascal Tyrrell, for answering my email last February and holding my interview on the same day as my Chemistry exam. Never before had I met such an – in a word – outstanding professor, and I dare say that I will never meet one like him throughout the rest of my academic journey.

Samantha Santoro

Posted on

A Medical Ethics ROP Journey with Jayun Bae

Jayun Bae – ROP299Y 2016-17
My name is Jayun Bae and I am completing my second year in the Neuroscience and Bioethics majors at the University of Toronto, St. George. I was a 2016-2017 Research Opportunity Program (ROP) student in Dr. Pascal Tyrrell’s lab, working on a study that investigated the ethics of sharing patient data with private organizations (see my timeline above). I am a member of the Hart House Debating Club and an events associate for the Life Science Student Network. 
                                                               
My ROP project was necessitated by the partnership proposed by the Medical image Networking Enterprise (MiNE) that would establish a data-sharing relationship between public and private sector organizations. The ethical concerns with the partnership involved patient consent, privacy, and financial gain – but there were also issues that I
uncovered throughout the project. It quickly became clear that the answers could not be found through an examination of precedence or legal documents, because many of the research actions that would take place (specifically involving private organizations) fell in the grey area between what was legal and what was ethical. For example, the Personal Information Protection and Electronic Documents Act (PIPEDA) and Personal Health Information Protection Act (PHIPA) are two guidelines for organizations to follow when handling patient data – but neither are able to clearly and positively dictate how this partnership should operate.
Therefore, I developed a study that would seek expert opinions through the administration of a survey. I conducted interviews at Sunnybrook Health Sciences Centre and the University of Toronto and performed qualitative data analysis. My ROP project was presented at the ROP Poster Fair and the Victoria College Research Day events. The ROP was an extremely valuable experience in gaining research skills, and I’m grateful to
Dr. Tyrrell for the guidance and mentorship. The project is not yet completed, so I am looking forward to continuing the study beyond the scope of the ROP.   Please have a look at my poster from the 2017 ROP Research Day below:
Posted on

MRI, Statistics, Carotid Arteries, and 1000 Cups of Coffee with George Wang

GeorgeWang – ROP299Y 2016-17
I’m George. I have recently completed my 2nd year undergrad at the University of Toronto studying physiology and physics. In the fall-winter term of 2016-17 I had the privilege to work in Pascal’s group, looking into carotid artery MRI and using the volume of the carotid artery vessel wall as a marker for atherosclerosis. Having an acquired interest in medical imaging and a previous summer position working with PET, I saw this as an excellent opportunity to expand my knowledge of the field while having the chance to be exposed to clinical research methods. Above is my account of how the year went in a nutshell.
 
Have a look at my poster from the ROP Research Day below…
 
 
Posted on

Engaging Primary Care in Research: Not Always an Easy Task

I am Stella Bing Xin Song, currently a second year student studying pharmacology and psychology at University of Toronto. I was fortunate to be a part of the 2016 Research Opportunity Program (ROP) supervised by Dr. Pascal Tyrrell in the Department of Medical Imaging at University of Toronto. 
My ROP project focused on evaluating the feasibility of using MRI as the primary imaging modality for carotid artery stenosis diagnosis and assessment (not sure what we are talking about? See previous posts here and here). Along with Ginni Ting, a student volunteer in Dr. Tyrrell’s lab, we surveyed physicians in the Niagara region of Ontario to learn about their perspectives on this proposal. Our community partner in this research was Heart Niagara – a fantastic local organization that has been guiding advances in cardiac health education and services since 1977.
Most of the responding physicians saw approximately 2000 or more patients per year. Physicians expressed a variety of care-related decisions for carotid artery stenosis patients, especially for those where diagnosis was less obvious with less than 70% stenosis. Most responding physicians would consider MRI over Ultrasound as the first-line diagnostic imaging modality, because of its ability to detect IPH yielding more pertinent information. IPH is bleeding within the plaques, which causes them to become more vulnerable (see vulnerable plaque). There is a 6 times greater risk of stroke in people with IPH! For those who were reluctant to consider it, they expressed that it was mostly due to their concerns for the relative cost and current wait time for MRI. 

Unfortunately, the response rate for this online survey was very low. Reasons given for the reluctance to participate were that physicians were on a tight schedule and were busy with their patients. Feedback from participants was that the online survey seemed long. Nevertheless, from the responses received, we were able to learn more about physicians’ perspectives of using MRI for carotid artery stenosis diagnosis and assessment.

In the end, it was an exciting and valuable experience to plan out and execute this research project. Most importantly, I had the pleasure to join Dr. Tyrrell’s lab and meet his team. I am grateful for all the help and support which I have received throughout my time at the lab. I look forward to continuing to work as a member of Dr. Tyrrell’s lab.

Stella Bing

Posted on

U of T Research Opportunity Program – Clare Sheen

Clare Sheen is an undergraduate student at the University of Toronto, in process of completing her Bachelor of Sciences in Genomics and Microbiology/Molecular Genetics. She was a 2015-6 Research Opportunity Program (ROP) student working on designing the Medical Image Network Enterprise (MiNE) interface for Dr. Pascal Tyrrell from U of T’s Department of Medical Imaging. She is currently a social director on the Life Science Student Network exec team and a volunteer at U of T’s Agrawal Lab where she helps with Drosophila experiments. She continues to seasonally work as a student camp teacher in the summer.

At the Research Opportunity Program (ROP) fair on March 3rd, U of T ROP students from different departments came together to share their research. A mock-up of the MiNE interface was presented in PowerPoint with the goal of increasing user engagement and encouraging the development of a medical imaging research community. Some features of the interface are presented below.

Posted on

Back to Basics… Midpoint Thoughts from an ROP Student

Reaching new heights? (Source: NYT)

Through the ‘Research Opportunity Program‘ (ROP) for second year students at U of T, I have been working on a project about physicians’ willingness to use MRI as the front-line diagnostic imaging technology for carotid stenosis patients. For a description see here.

After a recent discussion with Dr. Tyrrell (my supervisor), and as I approach the midpoint of my ROP project, I thought it would be a good idea to review some of my background knowledge of carotid stenosis from my work in the Fall term. Having a certain amount of independence while working on this project has been a great experience, but it also means I am responsible for keeping track of my own learning.


So, during the first week of January, I took out my notes, my Physiology textbook, and several articles in order to compile what I have learned so far and highlight areas that need further review.

Review in process!

Begrudgingly, I’ll admit that this ‘self-directed’ review process has shed new light on the usefulness of midterms in other courses. However, I still prefer this project-based review format. It has allowed me to review necessary information to make sure that it is fresh in my mind. Now I feel more prepared to begin the second half of the project. I’m looking forward to a major meeting this month and all the other exciting parts of the project to come.


Julia Robson

Posted on

Kiersten Thomas – Summer 2015 ROP at UofT: Another great student experience… part deux!

Kiersten Thomas – ROP summer 2015

Hello, I’m one of Professor Pascal Tyrrell’s summer ROP students and a second year Immunology Specialist student at the University of Toronto. I have just completed my summer research project investigating the cost-effectiveness of using MRA to evaluate asymptomatic carotid artery stenosis. 


I learned a lot and had an amazing time conducting research with Dr. Eli Lechtman and my ROP partner Indranil Balki. Using TreeAge Pro to create and analyse decision trees, we demonstrated that the additional information gained from an MRA/IPH scan increases the effectiveness of MRA when imaging carotid artery stenosis. 


Our research suggests that the additional information of the presence of intraplaque hemorrhage (IPH) would lead to better patient care, reducing long-term stroke risks. This means that MRA/IPH can be cost-effective or even dominant when compared to the current imaging strategy that uses standard ultrasound as a first-line imaging modality, for evaluating all degrees of asymptomatic carotid artery stenosis. Take a look at my timeline to see some of the highlights of my summer!


Thank you Dr. Pascal for this wonderful summer. 


Kiersten Thomas