Minimally invasive surgery is a procedure performed inside the body through small incisions and is referred to by different names depending on the region intervened. When it occurs in the abdominal or pelvic cavity, it is called laparoscopy. This procedure begins with the introduction of carbon dioxide into the abdominal cavity, with the aim of gently expanding it, separating the internal organs and providing better visibility and room for maneuvering. Then, an optical camera is introduced through one of the incisions, allowing real-time visualization of the operative field on an external monitor. The remaining cuts are used to introduce surgical instruments needed to cut, suture, and move tissues.

Home / Publications / Publication

Home / Publications / Publication

Laparoscopia
Image reproduced from the article.

AUGMENTED REALITY IN LAPAROSCOPY: A NEW WAY TO OPERATE COMFORTABLY

July 03, 25

Publication type: Article Summary
Original title: Laparoscopy with augmented reality adaptations
Article publication date: July 2020
Source: Journal of Biomedical Informatics
Authors: Ezequiel Zorzal, José Miguel Gomes, Maurício Sousa, Pedro Belchior, Pedro Silva, Nuno Figueiredo, Daniel Lopes & Joaquim Jorge

What is the goal, target audience, and areas of digital health it addresses?
     The main goal of the study is to develop and evaluate a hands-free augmented reality system that reduces surgeon fatigue and improves ergonomics, communication, and access to clinical data during laparoscopic surgery. The target audience includes surgeons as well as other members of the surgical team, hospital managers and healthcare decision makers, and even medical technology researchers and developers. The work addresses several areas of digital health, including augmented reality applied to surgery, medical ergonomics, multimodal interfaces and real-time clinical data visualization.

What is the context?
     Minimally invasive surgery is a procedure performed inside the body through small incisions and is referred to by different names depending on the region intervened. When it occurs in the abdominal or pelvic cavity, it is called laparoscopy. This procedure begins with the introduction of carbon dioxide into the abdominal cavity, with the aim of gently expanding it, separating the internal organs and providing better visibility and room for maneuvering. Then, an optical camera is introduced through one of the incisions, allowing real-time visualization of the operative field on an external monitor. The remaining cuts are used to introduce surgical instruments needed to cut, suture, and move tissues.

     Laparoscopy is commonly used in a variety of interventions, including: (1) removal of the gallbladder, appendix, ovarian cysts, or uterus; (2) treatment of hernias and pathologies of the intestines, kidneys, or bladder; and (3) bariatric surgeries to control obesity.

     Compared to open surgery, laparoscopy offers several advantages, such as less blood loss during the operation, reduced postoperative pain, lower risk of infection, shorter hospital stays, and a significantly faster recovery period.

What are the current approaches?
     In conventional laparoscopy, surgeons need to look away from the operative field and instruments to view the monitors, which compromises hand-eye coordination and hinders communication with the rest of the surgical team. These screens are often positioned at uncomfortable angles and at long distances, contributing to forced body postures that, when maintained during long farts, cause fatigue at the neck and shoulders. Other limitations include the high dependence on the visual interpretation of the projected data and the need for specialized technical training for effective use of the equipment.

     Augmented reality has emerged as a promising alternative to improve ergonomics, visualization of clinical information, and intraoperative communication. However, current applications available are largely based on the use of hand gestures or voice commands. These forms of interaction have significant limitations: gestures require an interruption of the surgical flow and can compromise the sterility of the field, as they require a pause to interact with the interface, while voice commands tend to be unreliable in noisy operating environments.

     In surgical settings, most augmented reality systems use optical see-through head-mounted displays, glasses that enable surgeons to view the operating room while virtual patient data is overlaid onto the real world. However, in laparoscopy, this approach is rarely applied. Instead, the common approach is highly immersive, with the surgical video displayed directly in front of the eyes, blocking most of the field of view, and impairing team communication. Additionally, current augmented reality solutions focus on the projection of 3D models and not on access to real clinical images in 2D, such as those obtained by magnetic resonance imaging or computed tomography. Thus, there is a need to develop more integrated, collaborative approaches adapted to the specificities of the surgical environment.

What does innovation consist of? How is the impact of this study assessed?
     This study developed an innovative augmented reality prototype designed to enhance the visualization of laparoscopic procedures, without requiring surgeons to adopt uncomfortable positions. The system is hands-free and features an intuitive multimodal interface that allows surgeons to view the laparoscopic video, interact with virtual patient images, and identify points of interest with precision.

     The prototype integrates two main hardware components: the Meta2 optical see-through augmented reality head-mounted display and surgical shoe adapted with a wireless HP Z3700 mouse that detects foot movements.

     The laparoscopic video is streamed live from a camera inserted into the patient and projected in the center of the augmented reality field of vision, following the user’s head movements, ensuring continuous visibility regardless of body posture. Additionally, two real 2D clinical images are presented laterally in the enlarged field, allowing quick visual comparisons and the choice of the most comfortable side for the visualization. As the images represent different planes of the abdominal or pelvic region, the system allows you to switch between them during the procedure. One method of navigating through images is by rotating your feet — moving backwards or forwards in images by turning left or right, respectively — or through the gaze, fixing the gaze on the virtual arrows displayed on screen. In this case, looking at the upward arrow advances to the next image, while looking at the downward arrow returns to the previous one. The detection of head orientation and gaze duration is carried out based on data collected by the headset’s accelerometer and gyroscope integrated into the glasses.

     The video feed and changes in patient image planes are synchronized in real time on the augmented reality headsets of all members of the surgical team, ensuring a shared augmented environment with situational awareness and clear communication. To ensure that everyone is focused on the same point, a pointing reticle can be activated using gaze — by looking 20° upward for a few seconds or by selecting a virtual button between the video and images. Each team member has a distinct pointer color, allowing quick identification of who is pointing.

     To evaluate the prototype’s usability and effectiveness, the study involved eight surgeons between the ages of 33 and 52 from the Champalimaud Foundation, all with at least seven years of experience in traditional laparoscopy, but limited experience with head-mounted displays. The sessions were conducted under dimmed lighting, simulating an operating room, with surgeons holding laparoscopic instruments while interacting with a pre-recorded laparoscopic video and anonymized magnetic resonance images. A think-aloud protocol was used, where surgeons explored the prototype while verbalizing their thoughts, allowing researchers to gather real-time insights about the experiment. After the session, participants completed the system usability scale, which rates perceived usability on a 0–100 scale and the NASA task load index, which evaluates perceived workload across six dimensions — mental, physical, and time demands, performance, effort, and frustration.

What are the main results? What is the future of this approach?
     The results were positive overall, indicating strong acceptance of the prototype’s features and its potential to improve laparoscopic surgery practice. The surgeons appreciated the ergonomic benefit of the video following head movements, allowing for continuous viewing and reducing cervical strain. The navigation through clinical images, either through the movement of the foot or by gaze fixation, was considered intuitive and complementary, highlighting the advantage of hands-free control to maintain the workflow during the procedure. Foot movement control was described as natural and familiar, due to its similarity to the pedals already used in surgery. Direct access to clinical images, without the need for external assistance, was perceived as a gain in autonomy. The gaze-based pointing mechanism proved to be effective for intraoperative communication, with participants valuing the clarity and visual distinction between the different cursors’. The system scored highly on the perceived usability scale (72.92), while the index results indicated workload levels within the appropriate parameters. Overall, users reported improved hand-eye coordination, more comfortable working postures, and enhanced team collaboration — while also reducing the learning curve and overcoming limitations of conventional laparoscopy.

     Some participants mentioned that laparoscopic video display was too small or lacked clarity and that the Meta2 augmented reality headset had a front-heavy weight distribution, causing some discomfort. For future versions, the adoption of the Microsoft HoloLens 2 is proposed, as it offers improved ergonomics, wireless operation, and a more balanced weight distribution. Another improvement would be the integration of stereoscopic 3D visualization, which presents slightly different images to the left and right eyes, simulating natural depth perception. The brain merges these images into a single three-dimensional representation, making it easier to interpret the position and movement of surgical instruments with greater accuracy and reduced cognitive effort. Although still an early-stage prototype, it demonstrated strong potential to advance the integration of augmented reality in minimally invasive surgery.

Would you like to know all the details?

Know more here

Do you have an innovative idea in healthcare field?

Share it with us and see it come to life.
We will help bring your projects to life!

Share my idea

Newsletter

Receive the latest updates from the InovarSaúde portal.

Subscribe

Support

FAQs

Contacts

User Manual

República Portuguesa logo
logotipo SNS
SPMS logotipo

Follow Us

YouTube
LinkedIn

Co-funded by

PRR Logotipo
república Portuguesa logo
União Europeia Logo

Newsletter

Receive the latest updates from the InovarSaúde portal.

Subscribe

Support

FAQs

Contacts

User Manual

República Portuguesa logo
SNS Logo
SPMS Logo

Follow Us

Co-funded by

PRR Logotipo
República Portuguesa logo
União Europeia Logo

Home / Publications / Publication

Laparoscopia
Image reproduced from the article.

AUGMENTED REALITY IN LAPAROSCOPY: A NEW WAY TO OPERATE COMFORTABLY

July 03, 25

Publication type: Article Summary
Original title: Laparoscopy with augmented reality adaptations
Article publication date: July 2020
Source: Journal of Biomedical Informatics
Authors: Ezequiel Zorzal, José Miguel Gomes, Maurício Sousa, Pedro Belchior, Pedro Silva, Nuno Figueiredo, Daniel Lopes & Joaquim Jorge

What is the goal, target audience, and areas of digital health it addresses?
     The main goal of the study is to develop and evaluate a hands-free augmented reality system that reduces surgeon fatigue and improves ergonomics, communication, and access to clinical data during laparoscopic surgery. The target audience includes surgeons as well as other members of the surgical team, hospital managers and healthcare decision makers, and even medical technology researchers and developers. The work addresses several areas of digital health, including augmented reality applied to surgery, medical ergonomics, multimodal interfaces and real-time clinical data visualization.

What is the context?
     Minimally invasive surgery is a procedure performed inside the body through small incisions and is referred to by different names depending on the region intervened. When it occurs in the abdominal or pelvic cavity, it is called laparoscopy. This procedure begins with the introduction of carbon dioxide into the abdominal cavity, with the aim of gently expanding it, separating the internal organs and providing better visibility and room for maneuvering. Then, an optical camera is introduced through one of the incisions, allowing real-time visualization of the operative field on an external monitor. The remaining cuts are used to introduce surgical instruments needed to cut, suture, and move tissues.

     Laparoscopy is commonly used in a variety of interventions, including: (1) removal of the gallbladder, appendix, ovarian cysts, or uterus; (2) treatment of hernias and pathologies of the intestines, kidneys, or bladder; and (3) bariatric surgeries to control obesity.

     Compared to open surgery, laparoscopy offers several advantages, such as less blood loss during the operation, reduced postoperative pain, lower risk of infection, shorter hospital stays, and a significantly faster recovery period.

What are the current approaches?
     In conventional laparoscopy, surgeons need to look away from the operative field and instruments to view the monitors, which compromises hand-eye coordination and hinders communication with the rest of the surgical team. These screens are often positioned at uncomfortable angles and at long distances, contributing to forced body postures that, when maintained during long farts, cause fatigue at the neck and shoulders. Other limitations include the high dependence on the visual interpretation of the projected data and the need for specialized technical training for effective use of the equipment.

     Augmented reality has emerged as a promising alternative to improve ergonomics, visualization of clinical information, and intraoperative communication. However, current applications available are largely based on the use of hand gestures or voice commands. These forms of interaction have significant limitations: gestures require an interruption of the surgical flow and can compromise the sterility of the field, as they require a pause to interact with the interface, while voice commands tend to be unreliable in noisy operating environments.

     In surgical settings, most augmented reality systems use optical see-through head-mounted displays, glasses that enable surgeons to view the operating room while virtual patient data is overlaid onto the real world. However, in laparoscopy, this approach is rarely applied. Instead, the common approach is highly immersive, with the surgical video displayed directly in front of the eyes, blocking most of the field of view, and impairing team communication. Additionally, current augmented reality solutions focus on the projection of 3D models and not on access to real clinical images in 2D, such as those obtained by magnetic resonance imaging or computed tomography. Thus, there is a need to develop more integrated, collaborative approaches adapted to the specificities of the surgical environment.

What does innovation consist of? How is the impact of this study assessed?
     This study developed an innovative augmented reality prototype designed to enhance the visualization of laparoscopic procedures, without requiring surgeons to adopt uncomfortable positions. The system is hands-free and features an intuitive multimodal interface that allows surgeons to view the laparoscopic video, interact with virtual patient images, and identify points of interest with precision.

     The prototype integrates two main hardware components: the Meta2 optical see-through augmented reality head-mounted display and surgical shoe adapted with a wireless HP Z3700 mouse that detects foot movements.

     The laparoscopic video is streamed live from a camera inserted into the patient and projected in the center of the augmented reality field of vision, following the user’s head movements, ensuring continuous visibility regardless of body posture. Additionally, two real 2D clinical images are presented laterally in the enlarged field, allowing quick visual comparisons and the choice of the most comfortable side for the visualization. As the images represent different planes of the abdominal or pelvic region, the system allows you to switch between them during the procedure. One method of navigating through images is by rotating your feet — moving backwards or forwards in images by turning left or right, respectively — or through the gaze, fixing the gaze on the virtual arrows displayed on screen. In this case, looking at the upward arrow advances to the next image, while looking at the downward arrow returns to the previous one. The detection of head orientation and gaze duration is carried out based on data collected by the headset’s accelerometer and gyroscope integrated into the glasses.

     The video feed and changes in patient image planes are synchronized in real time on the augmented reality headsets of all members of the surgical team, ensuring a shared augmented environment with situational awareness and clear communication. To ensure that everyone is focused on the same point, a pointing reticle can be activated using gaze — by looking 20° upward for a few seconds or by selecting a virtual button between the video and images. Each team member has a distinct pointer color, allowing quick identification of who is pointing.

     To evaluate the prototype’s usability and effectiveness, the study involved eight surgeons between the ages of 33 and 52 from the Champalimaud Foundation, all with at least seven years of experience in traditional laparoscopy, but limited experience with head-mounted displays. The sessions were conducted under dimmed lighting, simulating an operating room, with surgeons holding laparoscopic instruments while interacting with a pre-recorded laparoscopic video and anonymized magnetic resonance images. A think-aloud protocol was used, where surgeons explored the prototype while verbalizing their thoughts, allowing researchers to gather real-time insights about the experiment. After the session, participants completed the system usability scale, which rates perceived usability on a 0–100 scale and the NASA task load index, which evaluates perceived workload across six dimensions — mental, physical, and time demands, performance, effort, and frustration.

What are the main results? What is the future of this approach?
     The results were positive overall, indicating strong acceptance of the prototype’s features and its potential to improve laparoscopic surgery practice. The surgeons appreciated the ergonomic benefit of the video following head movements, allowing for continuous viewing and reducing cervical strain. The navigation through clinical images, either through the movement of the foot or by gaze fixation, was considered intuitive and complementary, highlighting the advantage of hands-free control to maintain the workflow during the procedure. Foot movement control was described as natural and familiar, due to its similarity to the pedals already used in surgery. Direct access to clinical images, without the need for external assistance, was perceived as a gain in autonomy. The gaze-based pointing mechanism proved to be effective for intraoperative communication, with participants valuing the clarity and visual distinction between the different cursors’. The system scored highly on the perceived usability scale (72.92), while the index results indicated workload levels within the appropriate parameters. Overall, users reported improved hand-eye coordination, more comfortable working postures, and enhanced team collaboration — while also reducing the learning curve and overcoming limitations of conventional laparoscopy.

     Some participants mentioned that laparoscopic video display was too small or lacked clarity and that the Meta2 augmented reality headset had a front-heavy weight distribution, causing some discomfort. For future versions, the adoption of the Microsoft HoloLens 2 is proposed, as it offers improved ergonomics, wireless operation, and a more balanced weight distribution. Another improvement would be the integration of stereoscopic 3D visualization, which presents slightly different images to the left and right eyes, simulating natural depth perception. The brain merges these images into a single three-dimensional representation, making it easier to interpret the position and movement of surgical instruments with greater accuracy and reduced cognitive effort. Although still an early-stage prototype, it demonstrated strong potential to advance the integration of augmented reality in minimally invasive surgery.

Would you like to know all the details?

Know more here
Literatura sobre os cuidados de saúde no futuro

What Literature Reveals About Healthcare in the Future

The healthcare sector is undergoing rapid transformation driven by population aging, increasing complexity of care, and digital advancements, in a context that requires greater integration, sustainability and adaptation to new realities such as the European…

Read more
Perturbação do sono

A Digital Intervention for Insomnia in Oncology

Insomnia is a sleep disorder characterised by persistent difficulties in initiating sleep, maintaining sleep during the night, or achieving restful sleep. These difficulties arise even in the presence of adequate sleeping conditions and are often…

Read more
Sistema de telemonitorização remota

Digital Technology Revolutionising Post-cardiac Surgery

According to the World Health Organisation, cardiovascular disease remains the leading cause of death worldwide, responsible for around 17.9 million deaths a year. Its high prevalence is associated with unhealthy lifestyles characterised by poor diet,…

Read more
Sistema robótico autónomo INSIDE

Autonomous Robotics System for Autism Therapy

Autism spectrum disorder is a neurodevelopmental condition with significant clinical, social and economic repercussions throughout life. According to the World Health Organization, it is estimated to affect approximately 1 in 160 children worldwide. Its origin…

Read more
Enfermeira com um telefone

Mobile Application to Improve Workflows in Nursing Homes

Portugal has one of the highest aging populations in the world, placing increasing pressure on elderly care services, especially in nursing homes. Healthcare professionals in these facilities are often overwhelmed due to the increasing number…

Read more
troca de informações de saúde e interoperabilidade

New Era of Interoperability in Healthcare Systems

The growing use of electronic health records, digital diagnostic systems and remote monitoring technologies has led to a significant increase in the volume and complexity of health data. This increase intensifies the need for continuous,…

Read more
robótica colaborativa

Collaborative Robotics Improves Working Conditions

Workers face growing challenges in the industrial environment. Among the most critical are fatigue and inappropriate postures, often associated with repetitive tasks and working conditions that lack ergonomic suitability. These factors represent significant risks for…

Read more
Benefícios da Eletrônica Médica

Detection of Anxiety and Panic Attacks in Real Time

The growing number of people with anxiety disorders, along with increased awareness of mental health, drives the need for new technological tools that provide remote and continuous monitoring of anxiety and panic disorders. Thus, the…

Read more
tele-ecografia

A Novel Approach for Robotic-assisted Tele-echography

Currently, robotic systems for ultrasound diagnostic procedures fall into two main categories: portable robots that require manual positioning and fully autonomous robotic systems that independently control the ultrasound probe’s orientation and positioning. Portable robots rely…

Read more
Personalização e tecnologia na gestão da Diabetes

Personalization and Technology in Diabetes Management

IPDM has significant potential to improve diabetes management and drive health system reforms to become high-performing, effective, equitable, accessible, and sustainable. Evidence and good practices inspire health system transformation. Adopting person-centred approaches like co-creation and…

Read more
TEF-HEALTH Logo

SPMS Integrates the TEF-Health Initiative

SPMS participates in the TEF-Health initiative as a partner in a consortium composed of 51 entities from 9 European Union countries. This action is co-financed by the European Commission and has a duration of five…

Read more
Global Digital Health Partnership Logo

SPMS Represents Portugal as Vice-president of GDHP

The GDHP is an intergovernmental organization in the digital health sector that facilitates cooperation and collaboration between government representatives and the World Health Organization (WHO). Its purpose is to foster policymaking that promote the digitalization…

Read more
Portugal INCoDe.2030

Digital Transformation of Health at INCoDe.2030 in Tomar

The “National Digital Skills Initiative e.2030, Portugal” (INCoDe.2030) is an initiative that aims to improve the Portuguese population’s level of digital skills, placing Portugal at the level of the most advanced European countries in this…

Read more
HealthData@PT Logo

HealthData@PT: New SPMS Initiative for Health Data

Action HealthData@PT is launched in the context of the implementation of the European Health Data Space, and is an initiative approved by the European Commission under the EU4Health 2021-2027 programme. This initiative contributes to the…

Read more

Do you have an innovative idea in healthcare field?

Share it with us and see it come to life.
We will help bring your projects to life!

Share my idea

Newsletter

Receive the latest updates from the Inovarsaúde portal.

Subscribe

Support

FAQs

Contacts

User Manual

República Portuguesa logo
SNS Logo
SPMS Logo

Follow Us

YouTube
LinkedIn

Co-funded by

PRR Logotipo
República Portuguesa logo
União Europeia Logo
Scroll to Top