Autism spectrum disorder is a neurodevelopmental condition with significant clinical, social and economic repercussions throughout life. According to the World Health Organization, it is estimated to affect approximately 1 in 160 children worldwide. Its origin is multifactorial, resulting from the interaction between genetic factors, such as mutations in genes associated with brain development, and environmental factors, such as pregnancy complications or prenatal exposure to neurotoxic substances. This condition is characterized by significant heterogeneity, with clinical manifestations ranging from severe deficits in communication and social interaction to more subtle difficulties, such as alterations in the functional use of language in everyday life. This diversity has led to the unification of previously distinct clinical classifications — such as infantile autism, Asperger’s syndrome or childhood disintegrative disorder — into autism spectrum disorder, as reflected in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders.

Home / Publications / Publication

Home / Publications / Publication

Sistema robótico autónomo INSIDE
Image reproduced from the INESC ID.

Publication type: Article Summary
Original title: Project INSIDE: towards autonomous semi-unstructured human–robot social interaction in autism therapy
Article publication date: May 2019
Source: GitHub
Authors: Francisco Melo, Alberto Sardinha, David Belo, Marta Couto, Miguel Faria, Anabela Farias, Hugo Gambôa, Cátia Jesus, Mithun Kinarullathil, Pedro Lima, Luís Luz, André Mateus, Isabel Melo, Plinio Moreno, Daniel Osório, Ana Paiva, Jhielson Pimentel, João Rodrigues, Pedro Sequeira, Rubén Solera-Ureña, Miguel Vasco, Manuela Veloso & Rodrigo Ventura

What is the goal, target audience, and areas of digital health it addresses?
     The study aims to develop and evaluate the INSIDE autonomous robotic system, designed to promote dynamic, adaptive and personalized social interactions for children diagnosed with autism spectrum disorder, in a real clinical context. The target audience includes health professionals involved in therapeutic contexts, researchers in the field of human-robot interaction and organizations dedicated to the development of digital technologies applied to health. Within the scope of digital health, it falls within the areas of social robotics, digital rehabilitation, artificial intelligence applied to clinical contexts and support technologies.

What is the context?
     Autism spectrum disorder is a neurodevelopmental condition with significant clinical, social and economic repercussions throughout life. According to the World Health Organization, it is estimated to affect approximately 1 in 160 children worldwide. Its origin is multifactorial, resulting from the interaction between genetic factors, such as mutations in genes associated with brain development, and environmental factors, such as pregnancy complications or prenatal exposure to neurotoxic substances.

     This condition is characterized by significant heterogeneity, with clinical manifestations ranging from severe deficits in communication and social interaction to more subtle difficulties, such as alterations in the functional use of language in everyday life. This diversity has led to the unification of previously distinct clinical classifications — such as infantile autism, Asperger’s syndrome or childhood disintegrative disorder — into autism spectrum disorder, as reflected in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders.

     Given the variability of these children’s sensory and cognitive profiles, therapeutic approaches must be carefully adapted to each case. In this context, the spontaneous interest that many of them show in predictable, non-human devices has driven the use of interactive technologies such as social robots, helping to create more accessible and less intimidating therapeutic contexts.

What are the current approaches?
     Currently, therapeutic interventions for children with autism spectrum disorder are based on multidisciplinary and personalized approaches, combining behavioural and educational models. Among the models with the most scientific evidence are Applied Behavior Analysis, the Early Start Denver Model and the Treatment and Education of Autistic and Related Communication Handicapped Children. These interventions are often complemented by specialized therapies, such as speech therapy, occupational therapy and psychomotor therapy. In cases with associated comorbidities, such as anxiety, attention deficit/hyperactivity disorder or sleep disorders, pharmacological intervention may be necessary, supervised by health professionals.

     At the same time, interactive digital technologies such as virtual reality and social robots have been explored as complementary tools. Robots such as NAO, QTrobot and KASPAR have proved effective in developing social and communication skills. However, most of these systems still rely on the Wizard-of-Oz model, where the robot is controlled by a human, limiting their autonomy and clinical applicability on a large scale.

What does innovation consist of? How is the impact of this study assessed?
     The innovation of this study consisted of the development of INSIDE, a fully autonomous robotic system designed to facilitate personalised social interactions with children with autism spectrum disorder in a real clinical environment. To enable independent therapeutic sessions, an integrated infrastructure was created that allows the ASTRO robot to adapt its behaviour in real time to the child’s responses and the guidelines defined by the clinical team.

     The INSIDE system’s hardware includes the ASTRO robot, 3D cameras, a tablet for interactive activities, microphones for voice commands and sensors on the robot to detect objects. To interpret the environment and adjust to the child’s actions, the system uses algorithms such as: Kalman filters that estimate the child’s position using data from the various cameras; Monte-Carlo localisation that allows the robot to locate itself in the room, based on the sensors and map of the room; and Markov models that recognise voice commands, guiding the robot’s response. The software architecture is hierarchical and made up of four modules: perception, which transforms sensor data into useful information; decision, which defines the robot’s behaviour based on this information; execution, which translates these decisions into actions; and supervision, which makes it possible to correct perceptions or decisions. In this way, therapists can monitor the behaviour of the robot and the child in real time, intervening only when necessary.

     When the robot enters the room, it greets the child in a personalised way, automatically positions itself in the space and proposes therapeutic activities such as searching for objects, solving puzzles, collaborating on tasks or playing turn-based games. It then explains the selected activity and encourages the child to take part. During interactions, the robot uses verbal language in a humanised voice, facial expressions animated on an LCD screen and physical movements adjusted to the context.

     To assess the impact of the INSIDE system, a therapeutic programme was carried out with 18 children diagnosed with autism spectrum disorder for 4 weeks at the Garcia de Orta Hospital in Almada, Portugal. The 121 therapeutic sessions, lasting between 15 and 25 minutes, were evaluated in three dimensions: functional autonomy, based on the robot’s ability to act without human intervention; quality of social interaction, based on the children’s spontaneous involvement; and clinical acceptance, through feedback from healthcare professionals. In addition, to collect physiological data, 4 of the children wore a t-shirt equipped with electrocardiogram sensors and accelerometers to measure their heart rate and movements during all the sessions.

What are the main results? What is the future of these technologies?
     The results demonstrated the effectiveness and applicability of the INSIDE system in a real clinical context. The ASTRO robot carried out all the sessions without the need for direct control by the technicians. Throughout the sessions, there was a decreasing frequency of human interventions, which demonstrated the system’s ability to adapt to the children’s responses and the demands of each interaction. The children remained consistently involved in the activities and showed socially relevant behaviours, such as reciprocity, imitation and the ability to respect the robot’s turn during the games. The health professionals highlighted the ease with which the robot could be integrated into clinical practice and its therapeutic relevance, while also emphasising the increase in motivation and participation on the part of the children. The analysis also showed that the children’s heart rate was generally higher during tasks with greater stimulation, such as ball games or situations with obstacles, which provides useful data for personalising the next therapy sessions.

     For the future, the system is expected to evolve by integrating more advanced models of emotional perception, natural language processing and behavioural pattern recognition, allowing for even deeper personalisation of interactions. INSIDE could be adapted to other contexts, such as schools and development support centres, and integrated into hybrid or remote intervention models through telehealth solutions, contributing to greater accessibility to specialised therapies.

Do you have an innovative idea in healthcare field?

Share it with us and see it come to life.
We will help bring your projects to life!

Newsletter

Receive the latest updates from the InovarSaúde portal.

Support

República Portuguesa logo
logotipo SNS
SPMS logotipo

Follow Us

Co-funded by

PRR Logotipo
república Portuguesa logo
União Europeia Logo

Newsletter

Receive the latest updates from the InovarSaúde portal.

Support

República Portuguesa logo
SNS Logo
SPMS Logo

Follow Us

Co-funded by

PRR Logotipo
República Portuguesa logo
União Europeia Logo

Home / Publications / Publication

Sistema robótico autónomo INSIDE
Image reproduced from the INESC ID.

Publication type: Article Summary
Original title: Project INSIDE: towards autonomous semi-unstructured human–robot social interaction in autism therapy
Article publication date: May 2019
Source: GitHub
Authors: Francisco Melo, Alberto Sardinha, David Belo, Marta Couto, Miguel Faria, Anabela Farias, Hugo Gambôa, Cátia Jesus, Mithun Kinarullathil, Pedro Lima, Luís Luz, André Mateus, Isabel Melo, Plinio Moreno, Daniel Osório, Ana Paiva, Jhielson Pimentel, João Rodrigues, Pedro Sequeira, Rubén Solera-Ureña, Miguel Vasco, Manuela Veloso & Rodrigo Ventura

What is the goal, target audience, and areas of digital health it addresses?
     The study aims to develop and evaluate the INSIDE autonomous robotic system, designed to promote dynamic, adaptive and personalized social interactions for children diagnosed with autism spectrum disorder, in a real clinical context. The target audience includes health professionals involved in therapeutic contexts, researchers in the field of human-robot interaction and organizations dedicated to the development of digital technologies applied to health. Within the scope of digital health, it falls within the areas of social robotics, digital rehabilitation, artificial intelligence applied to clinical contexts and support technologies.

What is the context?
     Autism spectrum disorder is a neurodevelopmental condition with significant clinical, social and economic repercussions throughout life. According to the World Health Organization, it is estimated to affect approximately 1 in 160 children worldwide. Its origin is multifactorial, resulting from the interaction between genetic factors, such as mutations in genes associated with brain development, and environmental factors, such as pregnancy complications or prenatal exposure to neurotoxic substances.

     This condition is characterized by significant heterogeneity, with clinical manifestations ranging from severe deficits in communication and social interaction to more subtle difficulties, such as alterations in the functional use of language in everyday life. This diversity has led to the unification of previously distinct clinical classifications — such as infantile autism, Asperger’s syndrome or childhood disintegrative disorder — into autism spectrum disorder, as reflected in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders.

     Given the variability of these children’s sensory and cognitive profiles, therapeutic approaches must be carefully adapted to each case. In this context, the spontaneous interest that many of them show in predictable, non-human devices has driven the use of interactive technologies such as social robots, helping to create more accessible and less intimidating therapeutic contexts.

What are the current approaches?
     Currently, therapeutic interventions for children with autism spectrum disorder are based on multidisciplinary and personalized approaches, combining behavioural and educational models. Among the models with the most scientific evidence are Applied Behavior Analysis, the Early Start Denver Model and the Treatment and Education of Autistic and Related Communication Handicapped Children. These interventions are often complemented by specialized therapies, such as speech therapy, occupational therapy and psychomotor therapy. In cases with associated comorbidities, such as anxiety, attention deficit/hyperactivity disorder or sleep disorders, pharmacological intervention may be necessary, supervised by health professionals.

     At the same time, interactive digital technologies such as virtual reality and social robots have been explored as complementary tools. Robots such as NAO, QTrobot and KASPAR have proved effective in developing social and communication skills. However, most of these systems still rely on the Wizard-of-Oz model, where the robot is controlled by a human, limiting their autonomy and clinical applicability on a large scale.

What does innovation consist of? How is the impact of this study assessed?
     The innovation of this study consisted of the development of INSIDE, a fully autonomous robotic system designed to facilitate personalised social interactions with children with autism spectrum disorder in a real clinical environment. To enable independent therapeutic sessions, an integrated infrastructure was created that allows the ASTRO robot to adapt its behaviour in real time to the child’s responses and the guidelines defined by the clinical team.

     The INSIDE system’s hardware includes the ASTRO robot, 3D cameras, a tablet for interactive activities, microphones for voice commands and sensors on the robot to detect objects. To interpret the environment and adjust to the child’s actions, the system uses algorithms such as: Kalman filters that estimate the child’s position using data from the various cameras; Monte-Carlo localisation that allows the robot to locate itself in the room, based on the sensors and map of the room; and Markov models that recognise voice commands, guiding the robot’s response. The software architecture is hierarchical and made up of four modules: perception, which transforms sensor data into useful information; decision, which defines the robot’s behaviour based on this information; execution, which translates these decisions into actions; and supervision, which makes it possible to correct perceptions or decisions. In this way, therapists can monitor the behaviour of the robot and the child in real time, intervening only when necessary.

     When the robot enters the room, it greets the child in a personalised way, automatically positions itself in the space and proposes therapeutic activities such as searching for objects, solving puzzles, collaborating on tasks or playing turn-based games. It then explains the selected activity and encourages the child to take part. During interactions, the robot uses verbal language in a humanised voice, facial expressions animated on an LCD screen and physical movements adjusted to the context.

     To assess the impact of the INSIDE system, a therapeutic programme was carried out with 18 children diagnosed with autism spectrum disorder for 4 weeks at the Garcia de Orta Hospital in Almada, Portugal. The 121 therapeutic sessions, lasting between 15 and 25 minutes, were evaluated in three dimensions: functional autonomy, based on the robot’s ability to act without human intervention; quality of social interaction, based on the children’s spontaneous involvement; and clinical acceptance, through feedback from healthcare professionals. In addition, to collect physiological data, 4 of the children wore a t-shirt equipped with electrocardiogram sensors and accelerometers to measure their heart rate and movements during all the sessions.

What are the main results? What is the future of these technologies?
     The results demonstrated the effectiveness and applicability of the INSIDE system in a real clinical context. The ASTRO robot carried out all the sessions without the need for direct control by the technicians. Throughout the sessions, there was a decreasing frequency of human interventions, which demonstrated the system’s ability to adapt to the children’s responses and the demands of each interaction. The children remained consistently involved in the activities and showed socially relevant behaviours, such as reciprocity, imitation and the ability to respect the robot’s turn during the games. The health professionals highlighted the ease with which the robot could be integrated into clinical practice and its therapeutic relevance, while also emphasising the increase in motivation and participation on the part of the children. The analysis also showed that the children’s heart rate was generally higher during tasks with greater stimulation, such as ball games or situations with obstacles, which provides useful data for personalising the next therapy sessions.

     For the future, the system is expected to evolve by integrating more advanced models of emotional perception, natural language processing and behavioural pattern recognition, allowing for even deeper personalisation of interactions. INSIDE could be adapted to other contexts, such as schools and development support centres, and integrated into hybrid or remote intervention models through telehealth solutions, contributing to greater accessibility to specialised therapies.

Enfermeira com um telefone

Mobile Application to Improve Workflows in Nursing Homes

Portugal has one of the highest aging populations in the world, placing increasing pressure on elderly care services, especially in nursing homes. Healthcare professionals in these facilities are often overwhelmed due to the increasing number…

Read more
troca de informações de saúde e interoperabilidade

New Era of Interoperability in Healthcare Systems

The growing use of electronic health records, digital diagnostic systems and remote monitoring technologies has led to a significant increase in the volume and complexity of health data. This increase intensifies the need for continuous,…

Read more
robótica colaborativa

Collaborative Robotics Improves Working Conditions

Workers face growing challenges in the industrial environment. Among the most critical are fatigue and inappropriate postures, often associated with repetitive tasks and working conditions that lack ergonomic suitability. These factors represent significant risks for…

Read more
Benefícios da Eletrônica Médica

Detection of Anxiety and Panic Attacks in Real Time

The growing number of people with anxiety disorders, along with increased awareness of mental health, drives the need for new technological tools that provide remote and continuous monitoring of anxiety and panic disorders. Thus, the…

Read more
tele-ecografia

A Novel Approach for Robotic-assisted Tele-echography

Currently, robotic systems for ultrasound diagnostic procedures fall into two main categories: portable robots that require manual positioning and fully autonomous robotic systems that independently control the ultrasound probe’s orientation and positioning. Portable robots rely…

Read more
Personalização e tecnologia na gestão da Diabetes

Personalization and Technology in Diabetes Management

IPDM has significant potential to improve diabetes management and drive health system reforms to become high-performing, effective, equitable, accessible, and sustainable. Evidence and good practices inspire health system transformation. Adopting person-centred approaches like co-creation and…

Read more
TEF-HEALTH Logo

SPMS Integrates the TEF-Health Initiative

SPMS participates in the TEF-Health initiative as a partner in a consortium composed of 51 entities from 9 European Union countries. This action is co-financed by the European Commission and has a duration of five…

Read more
Global Digital Health Partnership Logo

SPMS Represents Portugal as Vice-president of GDHP

The GDHP is an intergovernmental organization in the digital health sector that facilitates cooperation and collaboration between government representatives and the World Health Organization (WHO). Its purpose is to foster policymaking that promote the digitalization…

Read more
Portugal INCoDe.2030

Digital Transformation of Health at INCoDe.2030 in Tomar

The “National Digital Skills Initiative e.2030, Portugal” (INCoDe.2030) is an initiative that aims to improve the Portuguese population’s level of digital skills, placing Portugal at the level of the most advanced European countries in this…

Read more
HealthData@PT Logo

HealthData@PT: New SPMS Initiative for Health Data

Action HealthData@PT is launched in the context of the implementation of the European Health Data Space, and is an initiative approved by the European Commission under the EU4Health 2021-2027 programme. This initiative contributes to the…

Read more

Do you have an innovative idea in healthcare field?

Share it with us and see it come to life.
We will help bring your projects to life!

Newsletter

Receive the latest updates from the Inovarsaúde portal.

Support

FAQs

Contacts

República Portuguesa logo
SNS Logo
SPMS Logo

Follow Us

Co-funded by

PRR Logotipo
República Portuguesa logo
União Europeia Logo
Scroll to Top