Multi-Modal Learning for Ultrasound Imaging

Multi-modal learning leverages multiple information gained from different modalities for enhancing a better performance of detecting targets which a single modality could not have achieved. This concept, originating from machine learning, has been widely introduced in medical imaging, especially ultrasound imaging. In this project, we aim to review papers adopting multi-modal learning for ultrasound images and to identify useful algorithms or methods applicable to our future research.

Detection of Heterogeneity of CNNs

Convolutional Neural Networks (CNN) are highly dependent on an abundance of good-quality, labelled data for complete training. With small or overly heterogenous training sets, a CNN may report passable overall test accuracy, but perform poorly or inequitably for certain subgroups of data. Post-hoc cluster analysis of the feature space was shown to detect groups for which a CNN performs suboptimally, and that variation of CNN prediction is improved with greater data availability. Current work focuses on determining empirical benchmarks, model selection, and comparison of complementary methods (canonical correlation analysis). The full title of this project is “Cluster Analysis on the Detection of Heterogeneity of Convolutional Neural Networks”.

Deep Generative Machine Learning

The purpose is to create a production pipeline for the artificial generation of ultrasound datasets in MSK. The proposed pipeline will generate synthetic data with the possibility of producing different stages of development of the disorder. Similarity measurement and comparison methodologies between authentic and synthetic data will be explored. The full title of this paper is “Deep Generative Machine Learning to Obtain Synthetic Ultrasound Data  for Rare Disease Computer Vision Research”.

CAD/AI in Medical Imaging

Computer-aided diagnosis (CAD) is a technology that is rapidly becoming one of the most researched topics in radiology. The purpose of this systematic review is to describe and determine the facilitators and barriers to the introduction and adoption of Computer-aided diagnosis (CAD), a type of Artificial Intelligence (AI), by Radiologists in their workplace. We use literature covering the period from 2000 to 2020, and specific search strategies. The full title of this project is “Computer-Aided Diagnosis (CAD) and Artificial Intelligence (AI) in Clinical Practice: Systematic Review on the Barriers and Facilitators to CAD Integration in Radiology”.

Artificial Intelligence & Machine Learning

Unknown Class Detection

Conventional deep rural network(DNN)-based methods take it for granted that prediction classes are the same between training, evaluation and deployment phases. The purpose of this research is to develop a method to incorporate into conventional DNN models that will allow for the identification of possible unknown classes (also termed, out of distribution), applicable to any medical imaging modalities, such as brain CT/MRI image datasets.

Guiding Ultrasound Users

Working on creating an AI-driven solution to guide ultrasound users on how to perform the scan, characterization of the synovium on the ultrasound images, and analysis of multiple sclerosis and vasculitis on the MRI images.

Automatic Detection & Segmentation

The full title of this project is “Automatic detection and segmentation of hemarthrosis on hemophiliac patients using ultrasound”. The project consists of designing, testing and deploying deep learning methodologies for the detection and segmentation of hemarthrosis in the joint space (knee, elbow and ankle) of hemophiliac patients. Funded by Novo Nordisk A/S.

Data Science


MiDAS comprises three interconnected database modules sharing a common back end.

Industry Connections

16 Bit

16 Bit is a Toronto-based company founded by two radiology physicians with backgrounds in computer science and engineering. Using various world-class medical datasets, they design custom neural network architectures. With a unique combination of medical and technical expertise, 16 Bit focuses on solving the most impactful clinical problems facing medicine today.


AceAge is a healthcare technology company, creating intuitive products to ease the aging process and improve health outcomes. Their product Karie is a personal health companion that organizes, schedules, and dispenses pills with one-button technology, ensuring that patients are taking the right medication at the right time.


The aim of this initiative is to (1) establish a research agreement and network between the Costa Rican healthcare system and Canadian medical imaging research (MiDATA – UofT) to share and transfer data and knowledge; and (2) create a research environment and develop the methodological pipeline and associated tools required for transformative AI solutions for clinical applications involving medical imaging.

IBM Watson Health

IBM works to enable healthcare technology to help organizations gain more insights to their data and simplify their operations. IBM Watson Health bridges data, technology and expertise together to transform healthcare. This allows for efficient, resilient and robust institutions . They provide solutions for healthcare providers, government, life sciences, diagnostic imaging, healthcare payers, and employers.


The Medical image Network Enterprise (MiNE) is a scalable, community-defined e-infrastructure with the primary objective of housing an electronic image-based inventory and data warehouse. MiNE also provides related tools for research and education, to support and encourage the University of Toronto research community. MiNE advocates the efficient use and privacy-sensitive sharing of existing research and clinical image data.

Nippon Steel

Nippon Steel looks to pursue world-leading technologies and manufacturing capabilities, and contribute to society with their products and services. They create and design various products like plates, flat products, bar and wire rod, construction products, pipes and tubes, stainless steel, titanium, railway, automotive and machinery parts.

Novo Nordisk

Novo Nordisk discovers and develops innovative biological medicines and makes them accessible to patients throughout the world. Their purpose is to drive change to defeat diabetes and other serious chronic diseases such as obesity, and rare blood and rare endocrine diseases by engineering scientific breakthroughs, expanding access to our medicines and working to prevent and ultimately cure the diseases they treat.

SofTx Innovations

SofTx Innovations is a software development startup company who builds deep tech applications, products, and platforms to integrate and leverage AI/ML advances for healthcare. As AI is a disruptive technology that provides unique insights, they help organizations identify where opportunities for continuous improvement exist in the areas of quality, productivity, and cost reduction across the organization.