Tele-sense
DURATION
2023.09 ~ Present
2 years 4 months
(ongoing)
KEY WORDS
HRI
Telehpatic
Contribution
Team Leader
Haptic System Design
Robot Engineering
Tele-operated avatar robots serve as a physical extension of the operator, facilitating active and meaningful communication with local users. Recognizing that non-verbal cues are critical for shaping user experience, our system is engineered to deliver rich expressive behaviors. It integrates HMD-based eye tracking to map the operator’s dynamic gaze in real-time, a tele-haptics interface specialized for social touch, and an AI-driven voice modulation system to maintain a consistent character persona. By synchronizing these multimodal signals, the system significantly enhances the naturalness of the interaction, thereby increasing user immersion and engagement.
Indendent Research
|
2023.09 - Present
Outcome
Achievement
[Paper] Hoseok Jung, et al. 2026. Affective Robotic Avatar: Enhancing Social Telepresence via Haptics and Expressive Cues. In Proceedings of HCI International 2026 (HCII ‘26). (Invited Full Paper, Session: Frontiers of HCI in XR)
Background
With the increasing adoption of robotic
technologies in healthcare, interest has grown in leveraging robots to support pediatric patients
through rehabilitation and emotional interaction.
Problem
However, most existing teleoperated robots remain focused on simple, conversation-based interactions, with limited research on more complex tasks such as grasping or object handover.
Solution
This project aims to develop a tele-haptic robot system that enables rich and diverse remote interactions between medical professionals and pediatric patients.
Service Application Design: Empathic Remote Avatar Robot for Non-verbal Connection and Social Touch
We developed avatar robot to build an emotional bond between counselors and clients beyond physical separation. The system resolves the lack of non-verbal cues—a common limitation in robotic counseling—by mirroring the operator's fine head movements and gaze, and by executing 'social touches' like handshakes or patting appropriate to the situation. This allows clients to engage in counseling without feeling a sense of incongruity, enabling a deeply empathetic interaction.
Key Interaction
Improve Nonverbal Communication
Paralinguistic
Voice Expression
Eye Gaze
Interaction
Embodied
Gesture Movement
Tele-haptic
Social Touch
Key Features
Paralinguistic Voice Expression
In interactions with a teleoperated robot, we considered that users are not engaging with a human operator, but with a robot character.
Therefore, the operator’s voice is not transmitted as-is; instead, it is transformed to create the experience of the robot character speaking directly.
Eye Gaze Interaction
The remote operator’s gaze was directly mapped onto the robot’s eyes.
As a result, the robot’s eye movements reflect the operator’s real-time visual attention, enabling natural and expressive non-verbal interaction.
Embodied Gesture Movement
The robot’s upper-body movements are controlled through the user’s physical motions, using inverse kinematics and hand tracking.
Both the neck and arms are mapped to the user’s movements, allowing the robot to mirror the operator’s upper-body behavior in real time.
Tele-haptic Social Touch
We configured a tele-haptic system by integrating a robot gripper with a fingertip haptic interface to deliver the sensation of physical contact, such as handmassage or patting. Beyond simple manipulation, this setup allows operators to physically perceive the grasping force during interaction, enabling authentic tactile communication and emotional connection through the robot.
Playful Haptic Interaction: Improving Robot Acceptance via Massage
We explore the potential effects of robot-mediated physical massage on users’ psychological barriers, approaching this study with the assumption that such tendencies may emerge. Specifically, we hypothesize that the experience may be perceived as a “fun and unique experience,” fostering a state of positive arousal that could create a more welcoming interaction atmosphere. This, in turn, is expected to enhance conversational engagement and facilitate more rapid rapport building with the robot.
Role and Responsibility