Breast-conserving surgery aims to remove tumors while preserving as much healthy breast tissue as possible, ensuring optimal aesthetic outcomes that are critical for a patient’s quality of life. To achieve this objective, precise location of the tumor is necessary during surgery, which is normally performed with the patient lying face-up (supine position). To plan the procedure, surgeons rely on medical imaging technologies that often provide limited perspectives due to the conditions under which the images are captured. For instance, MRI scans are performed with the patient lying face down (prone position), causing the breast to compress and deform, while 3D scans capture the natural shape of the breast in the upright position. Since neither imaging method reflects the breast’s actual position during surgery, merging internal radiological images (e.g., MRI) with external surface images (e.g., 3D scans) into a unified model is essential to create a realistic representation of the breast.

Home / Publications / Publication

Home / Publications / Publication

Rastreio do cancro da mama
Image reproduced from the article.

IMPROVEMENT IN BREAST TUMOR LOCALIZATION WITH AN IMAGE FUSION ALGORITHM

January 02, 25

Publication Type: Article Summary
Original title: 3D digital breast cancer models with multimodal fusion algorithms
Article publication date: February 2020
Source: Repositório da Universidade Nova de Lisboa
Authors: Sílvia Bessa, Pedro Gouveia, Pedro Carvalho, Cátia Rodrigues, Nuno Silva, Fátima Cardoso, Jaime Cardoso, Hélder Oliveira & Maria João Cardoso

What is the goal, target audience, and areas of digital health it addresses?
     The main goal of this research is to validate a fusion algorithm that integrates Magnetic Resonance Imaging (MRI) and 3D scan to improve tumor localization in breast cancer patients. The target audience includes surgeons, radiologists, and clinical researchers. The study addresses the areas of medical imaging, image processing, computer-assisted surgery, and augmented reality for surgery.

What is the context?
     Breast-conserving surgery aims to remove tumors while preserving as much healthy breast tissue as possible, ensuring optimal aesthetic outcomes that are critical for a patient’s quality of life. To achieve this objective, precise location of the tumor is necessary during surgery, which is normally performed with the patient lying face-up (supine position).

     To plan the procedure, surgeons rely on medical imaging technologies that often provide limited perspectives due to the conditions under which the images are captured. For instance, MRI scans are performed with the patient lying face down (prone position), causing the breast to compress and deform, while 3D scans capture the natural shape of the breast in the upright position. Since neither imaging method reflects the breast’s actual position during surgery, merging internal radiological images (e.g., MRI) with external surface images (e.g., 3D scans) into a unified model is essential to create a realistic representation of the breast. However, aligning these datasets is challenging not only because the images are captured in different positions but also due to the limited availability of anatomical reference points for alignment. To solve this issue, artificial reference points, such as marks made with black markers, can be used; however, these points are not visible on MRI, as it only captures internal structures. Alternatively, cod liver oil pills offer a low-cost and readily available solution, providing contrast on MRI due to their fat content.

What are the current approaches?
     Current approaches to breast image fusion typically focus on aligning radiological or surface imaging data independently, rather than integrating both into a unified 3D model.

     One approach used is the biomechanical models, which simulate how breast tissue moves and changes shape between different positions. These models are based on mathematical equations to predict tissue behavior in detail, generating realistic simulations. However, they are computationally intensive, prone to introducing distortions in breast shape, and often unable to reflect the unique properties of each patient’s breast.

     Another method involves non-physical models like Free Form Deformation (FFD), which use a flexible “grid” of control points to adjust the shape of images. This technique directly bends and stretches images to align them, offering a faster and simpler alternative to biomechanical models. However, FFD has limitations in terms of physical realism and may generate less accurate results as it does not consider the natural behavior of breast tissue.

     Combining these two methods—using biomechanical models for more realistic simulations and FFD for alignment adjustments—shows promise, but still faces challenges, such as limited clinical validation.

What does innovation consist of? How is the impact of this study assessed?
     The innovation in this study lies in the development and validation of a fusion algorithm that integrates MRI data and 3D scan data to create unified, patient-specific 3D breast models, intended to support surgical planning.

     The protocol begins with data acquisition, where both MRI scans and 3D scans are collected. Then, radiologists use the Horos R software to identify and outline tumors in the MRI scans, calculating their volume to create a 3D tumor model that is, then, overlaid onto the original MRI dataset. To prepare for fusion, two MRI datasets are created with the tumor marked: MRIprone (an image with the original MRI without biomechanical processing) and MRIup (an image adjusted with a biomechanical model to simulate the upright position obtained in the 3D scan).

     Both MRIprone and MRIup datasets are submitted to the fusion algorithm, which integrates them with the 3D scans using two types of alignment strategies: single breast fusion, where each breast is fused independently with the corresponding side of the breast in the 3D scan, and full torso fusion, where both breasts and the torso are fused as a single unit. The alignment performed involves superimposing the contours of both sets of images. This step relies on anatomical reference points as well as artificial reference points, known as Breast Surface Markers (BSM). These points are created using either black permanent markers on the breast and torso before 3D scan acquisition or cod liver oil pills placed in the same positions before MRI acquisition, ensuring correspondence between the two imaging modalities.

     Then, the fusion algorithm applies FFD adjustments to compensate for residual differences in breast shape caused by factors such as variations in imaging positions, soft tissue deformations, and gravitational effects. During FFD, the tumor location, initially outlined in the MRI data, is carefully adjusted to align with the fused 3D breast model.

     Seven breast cancer patients eligible for breast-conserving surgery at the Champalimaud Clinical Center in Portugal participated in the validation of the fusion algorithm with the application of BSM. The aim was also to identify the most effective approach for tumor detection by the algorithm, comparing the fusion of 3D scans with data obtained in MRIprone and MRIup, as well as individual breast fusion with full torso fusion alignment.

     The performance of the technology was evaluated using two complementary methods: Target Registration Error (TRE) and qualitative tumor localization validation. TRE quantifies the alignment accuracy between the MRI and 3D scan by measuring the Euclidean distance—a straight-line distance in 3D space—between corresponding BSM identified in both datasets, providing an objective metric for fusion precision. Additionally, qualitative validation was conducted by a breast surgeon, who compared the tumor’s location in the fused 3D breast model against clinical records, surgical annotations, and visible skin markers to ensure the tumor’s position was accurately represented in the final model.

What are the main results? What is the impact of these results? What is the future of this technology?
     The fusion algorithm successfully created accurate 3D breast models and demonstrated robust performance across different patient anatomies, as it was not significantly influenced by variations in breast volume.

     The results showed that single breast fusion with MRIprone provided more precise tumor localization despite having slightly higher TRE values, indicating a small mismatch in aligning the datasets. Specifically, the TRE for single breast fusion with MRIprone was 26.26 ± 6.61 mm, compared to a lower TRE of 18.5 ± 3.88 mm for single breast fusion with MRIup. However, the use of a biomechanical model in MRIup introduced artifacts such as axial elongation and lateral displacement, which negatively impacted tumor positioning accuracy. Qualitative analysis further highlighted that tumor localization was more accurate in 80% of cases with MRIprone. In contrast, the full torso fusion approaches exhibited the poorest overall performance.

     The results pave the way for more patient-specific breast cancer imaging in clinical practice. The validated algorithm eliminates the need for complex biomechanical models, offering a simpler solution for tumor localization. This approach supports automation, improves preoperative planning, and enhances surgical precision in breast-conserving treatments. The ability to generate precise 3D models also enables the integration of augmented reality, providing surgeons with real-time visualization of tumor location during operations, which should improve patient outcomes.

     The future involves obtaining MRI images in the supine position, bringing imaging acquisition conditions closer to surgical scenarios, which will allow for improved tumor localization accuracy. It is also necessary to expand datasets with more participants and integrate machine learning to enhance algorithm performance and enable the prediction of breast shape transformations. The use of augmented reality glasses, with the overlay of 3D breast models in real-world contexts, could revolutionize surgery by allowing direct visualization of tumor locations, making procedures more precise and efficient.

Would you like to know all the details?

Know more here

Do you have an innovative idea in healthcare field?

Share it with us and see it come to life.
We will help bring your projects to life!

Share my idea

Newsletter

Receive the latest updates from the InovarSaúde portal.

Subscribe

Support

FAQs

Contacts

República Portuguesa logo
logotipo SNS
SPMS logotipo

Follow Us

Co-funded by

PRR Logotipo
república Portuguesa logo
União Europeia Logo

Newsletter

Receive the latest updates from the InovarSaúde portal.

Subscribe

Support

FAQs

Contacts

República Portuguesa logo
SNS Logo
SPMS Logo

Follow Us

Co-funded by

PRR Logotipo
República Portuguesa logo
União Europeia Logo

Home / Publications / Publication

Rastreio do cancro da mama
Image reproduced from the article.

IMPROVEMENT IN BREAST TUMOR LOCALIZATION WITH AN IMAGE FUSION ALGORITHM

January 02, 25

Publication Type: Article Summary
Original title: 3D digital breast cancer models with multimodal fusion algorithms
Article publication date: February 2020
Source: Repositório da Universidade Nova de Lisboa
Authors: Sílvia Bessa, Pedro Gouveia, Pedro Carvalho, Cátia Rodrigues, Nuno Silva, Fátima Cardoso, Jaime Cardoso, Hélder Oliveira & Maria João Cardoso

What is the goal, target audience, and areas of digital health it addresses?
     The main goal of this research is to validate a fusion algorithm that integrates Magnetic Resonance Imaging (MRI) and 3D scan to improve tumor localization in breast cancer patients. The target audience includes surgeons, radiologists, and clinical researchers. The study addresses the areas of medical imaging, image processing, computer-assisted surgery, and augmented reality for surgery.

What is the context?
     Breast-conserving surgery aims to remove tumors while preserving as much healthy breast tissue as possible, ensuring optimal aesthetic outcomes that are critical for a patient’s quality of life. To achieve this objective, precise location of the tumor is necessary during surgery, which is normally performed with the patient lying face-up (supine position).

     To plan the procedure, surgeons rely on medical imaging technologies that often provide limited perspectives due to the conditions under which the images are captured. For instance, MRI scans are performed with the patient lying face down (prone position), causing the breast to compress and deform, while 3D scans capture the natural shape of the breast in the upright position. Since neither imaging method reflects the breast’s actual position during surgery, merging internal radiological images (e.g., MRI) with external surface images (e.g., 3D scans) into a unified model is essential to create a realistic representation of the breast. However, aligning these datasets is challenging not only because the images are captured in different positions but also due to the limited availability of anatomical reference points for alignment. To solve this issue, artificial reference points, such as marks made with black markers, can be used; however, these points are not visible on MRI, as it only captures internal structures. Alternatively, cod liver oil pills offer a low-cost and readily available solution, providing contrast on MRI due to their fat content.

What are the current approaches?
     Current approaches to breast image fusion typically focus on aligning radiological or surface imaging data independently, rather than integrating both into a unified 3D model.

     One approach used is the biomechanical models, which simulate how breast tissue moves and changes shape between different positions. These models are based on mathematical equations to predict tissue behavior in detail, generating realistic simulations. However, they are computationally intensive, prone to introducing distortions in breast shape, and often unable to reflect the unique properties of each patient’s breast.

     Another method involves non-physical models like Free Form Deformation (FFD), which use a flexible “grid” of control points to adjust the shape of images. This technique directly bends and stretches images to align them, offering a faster and simpler alternative to biomechanical models. However, FFD has limitations in terms of physical realism and may generate less accurate results as it does not consider the natural behavior of breast tissue.

     Combining these two methods—using biomechanical models for more realistic simulations and FFD for alignment adjustments—shows promise, but still faces challenges, such as limited clinical validation.

What does innovation consist of? How is the impact of this study assessed?
     The innovation in this study lies in the development and validation of a fusion algorithm that integrates MRI data and 3D scan data to create unified, patient-specific 3D breast models, intended to support surgical planning.

     The protocol begins with data acquisition, where both MRI scans and 3D scans are collected. Then, radiologists use the Horos R software to identify and outline tumors in the MRI scans, calculating their volume to create a 3D tumor model that is, then, overlaid onto the original MRI dataset. To prepare for fusion, two MRI datasets are created with the tumor marked: MRIprone (an image with the original MRI without biomechanical processing) and MRIup (an image adjusted with a biomechanical model to simulate the upright position obtained in the 3D scan).

     Both MRIprone and MRIup datasets are submitted to the fusion algorithm, which integrates them with the 3D scans using two types of alignment strategies: single breast fusion, where each breast is fused independently with the corresponding side of the breast in the 3D scan, and full torso fusion, where both breasts and the torso are fused as a single unit. The alignment performed involves superimposing the contours of both sets of images. This step relies on anatomical reference points as well as artificial reference points, known as Breast Surface Markers (BSM). These points are created using either black permanent markers on the breast and torso before 3D scan acquisition or cod liver oil pills placed in the same positions before MRI acquisition, ensuring correspondence between the two imaging modalities.

     Then, the fusion algorithm applies FFD adjustments to compensate for residual differences in breast shape caused by factors such as variations in imaging positions, soft tissue deformations, and gravitational effects. During FFD, the tumor location, initially outlined in the MRI data, is carefully adjusted to align with the fused 3D breast model.

     Seven breast cancer patients eligible for breast-conserving surgery at the Champalimaud Clinical Center in Portugal participated in the validation of the fusion algorithm with the application of BSM. The aim was also to identify the most effective approach for tumor detection by the algorithm, comparing the fusion of 3D scans with data obtained in MRIprone and MRIup, as well as individual breast fusion with full torso fusion alignment.

     The performance of the technology was evaluated using two complementary methods: Target Registration Error (TRE) and qualitative tumor localization validation. TRE quantifies the alignment accuracy between the MRI and 3D scan by measuring the Euclidean distance—a straight-line distance in 3D space—between corresponding BSM identified in both datasets, providing an objective metric for fusion precision. Additionally, qualitative validation was conducted by a breast surgeon, who compared the tumor’s location in the fused 3D breast model against clinical records, surgical annotations, and visible skin markers to ensure the tumor’s position was accurately represented in the final model.

What are the main results? What is the impact of these results? What is the future of this technology?
     The fusion algorithm successfully created accurate 3D breast models and demonstrated robust performance across different patient anatomies, as it was not significantly influenced by variations in breast volume.

     The results showed that single breast fusion with MRIprone provided more precise tumor localization despite having slightly higher TRE values, indicating a small mismatch in aligning the datasets. Specifically, the TRE for single breast fusion with MRIprone was 26.26 ± 6.61 mm, compared to a lower TRE of 18.5 ± 3.88 mm for single breast fusion with MRIup. However, the use of a biomechanical model in MRIup introduced artifacts such as axial elongation and lateral displacement, which negatively impacted tumor positioning accuracy. Qualitative analysis further highlighted that tumor localization was more accurate in 80% of cases with MRIprone. In contrast, the full torso fusion approaches exhibited the poorest overall performance.

     The results pave the way for more patient-specific breast cancer imaging in clinical practice. The validated algorithm eliminates the need for complex biomechanical models, offering a simpler solution for tumor localization. This approach supports automation, improves preoperative planning, and enhances surgical precision in breast-conserving treatments. The ability to generate precise 3D models also enables the integration of augmented reality, providing surgeons with real-time visualization of tumor location during operations, which should improve patient outcomes.

     The future involves obtaining MRI images in the supine position, bringing imaging acquisition conditions closer to surgical scenarios, which will allow for improved tumor localization accuracy. It is also necessary to expand datasets with more participants and integrate machine learning to enhance algorithm performance and enable the prediction of breast shape transformations. The use of augmented reality glasses, with the overlay of 3D breast models in real-world contexts, could revolutionize surgery by allowing direct visualization of tumor locations, making procedures more precise and efficient.

Would you like to know all the details?

Know more here
Sistema robótico autónomo INSIDE

Autonomous Robotics System for Autism Therapy

Autism spectrum disorder is a neurodevelopmental condition with significant clinical, social and economic repercussions throughout life. According to the World Health Organization, it is estimated to affect approximately 1 in 160 children worldwide. Its origin…

Read more
Enfermeira com um telefone

Mobile Application to Improve Workflows in Nursing Homes

Portugal has one of the highest aging populations in the world, placing increasing pressure on elderly care services, especially in nursing homes. Healthcare professionals in these facilities are often overwhelmed due to the increasing number…

Read more
troca de informações de saúde e interoperabilidade

New Era of Interoperability in Healthcare Systems

The growing use of electronic health records, digital diagnostic systems and remote monitoring technologies has led to a significant increase in the volume and complexity of health data. This increase intensifies the need for continuous,…

Read more
robótica colaborativa

Collaborative Robotics Improves Working Conditions

Workers face growing challenges in the industrial environment. Among the most critical are fatigue and inappropriate postures, often associated with repetitive tasks and working conditions that lack ergonomic suitability. These factors represent significant risks for…

Read more
Benefícios da Eletrônica Médica

Detection of Anxiety and Panic Attacks in Real Time

The growing number of people with anxiety disorders, along with increased awareness of mental health, drives the need for new technological tools that provide remote and continuous monitoring of anxiety and panic disorders. Thus, the…

Read more
tele-ecografia

A Novel Approach for Robotic-assisted Tele-echography

Currently, robotic systems for ultrasound diagnostic procedures fall into two main categories: portable robots that require manual positioning and fully autonomous robotic systems that independently control the ultrasound probe’s orientation and positioning. Portable robots rely…

Read more
Personalização e tecnologia na gestão da Diabetes

Personalization and Technology in Diabetes Management

IPDM has significant potential to improve diabetes management and drive health system reforms to become high-performing, effective, equitable, accessible, and sustainable. Evidence and good practices inspire health system transformation. Adopting person-centred approaches like co-creation and…

Read more
TEF-HEALTH Logo

SPMS Integrates the TEF-Health Initiative

SPMS participates in the TEF-Health initiative as a partner in a consortium composed of 51 entities from 9 European Union countries. This action is co-financed by the European Commission and has a duration of five…

Read more
Global Digital Health Partnership Logo

SPMS Represents Portugal as Vice-president of GDHP

The GDHP is an intergovernmental organization in the digital health sector that facilitates cooperation and collaboration between government representatives and the World Health Organization (WHO). Its purpose is to foster policymaking that promote the digitalization…

Read more
Portugal INCoDe.2030

Digital Transformation of Health at INCoDe.2030 in Tomar

The “National Digital Skills Initiative e.2030, Portugal” (INCoDe.2030) is an initiative that aims to improve the Portuguese population’s level of digital skills, placing Portugal at the level of the most advanced European countries in this…

Read more
HealthData@PT Logo

HealthData@PT: New SPMS Initiative for Health Data

Action HealthData@PT is launched in the context of the implementation of the European Health Data Space, and is an initiative approved by the European Commission under the EU4Health 2021-2027 programme. This initiative contributes to the…

Read more

Do you have an innovative idea in healthcare field?

Share it with us and see it come to life.
We will help bring your projects to life!

Share my idea

Newsletter

Receive the latest updates from the Inovarsaúde portal.

Subscribe

Support

FAQs

Contacts

República Portuguesa logo
SNS Logo
SPMS Logo

Follow Us

Co-funded by

PRR Logotipo
República Portuguesa logo
União Europeia Logo
Scroll to Top