Cordial Logo

CoRDial

Counterfactual Reasoning for Robotic Dialogue

This project will seek to enhance the care of older adults through the development of more intuitive and transparent human-robot interactions in care tasks.

Project Summary

This project successfully developed a multi-agent conversational framework designed to enhance human-robot interactions in elder care settings. Through seven co-design workshops with older adults and care workers in urban, rural, and respite care environments, we gathered valuable insights into user needs, leading to key refinements in our approach. The main findings from the workshop were around language/vocabulary, personalisation, data protection and safety.

Our framework, built using OpenAI services and Langchain, features a routing manager based on prototypical networks that dynamically assigns user queries to the most relevant task agent. These advancements contribute to making conversational AI more adaptive, transparent, and user-centered, enhancing its effectiveness in elder care settings.

Co-Creation Workshops

We conducted a series of workshops to gather insights from diverse environments and perspectives:

Workshop Details

  • Urban Engagement: Two workshops were held in Aberdeen, focusing on older adults from city areas, where we explored their preferences and needs in elder care robotics.
  • Rural and Remote Engagement: Two workshops were conducted in the Orkney Islands, involving older adults and care workers from rural and remote regions. This provided insights into the unique challenges faced in less accessible locations.
  • Respite Care: A workshop in North Berwick engaged older adults and care workers from respite care environments, giving us valuable input from both caregivers and those receiving temporary care.
  • Online Workshops: Our last two workshops were held online. One, which was held in November 2024, focused on showcasing our enhanced conversational framework with scenarios and gathering feedback to refine it. In our most recent workshop, held in January 2025, we had an engaging discussion about additional tasks we could explore and how participants expect the robot to perform them.

Key Findings

Personalisation is key, conversations and tasks should feel tailored to the individual (Some people doesn't like chit chat etc)
Engage in off-topic conversations to keep the interaction natural and flowing
Emotional intelligence is important — the robot should detect when something is off with the person (ex: sensing they are unwell) and respond appropriately.
Ensure understanding by asking: "Do you understand?" after explaining medications or important details.
Provide explanations for why certain medications or tasks are necessary.
Use simple and familiar terms for medications, like "blue pill" instead of specific medication names (ex. Panadol).
Co-Creation Activities Map

Architecture

Project Architecture Diagram

CoRDial offers a flexible and modular system for integrating new agents as tasks evolve. The Dialogue Manager oversees interactions, routing queries to specialized agents using the ProtoNet Router. This ensures effective management of user requests while maintaining context and handling ethical considerations. The Ethical Reviewer ensures compliance with healthcare guidelines. Task agents, l ike Off-Topic Question Answering and Medication Reminders, operate with unique personas and access relevant APIs.

Each agent has dedicated episodic memory for retaining conversation histories and providing context. The ProtoNet Router uses few-shot learning to classify user responses, ensuring efficient agent routing with improved interpretability. The system also integrates continuous learning, adapting based on user feedback stored in Firebase Cloud Firestore to enhance routing accuracy and overall conversational quality over time.

The CoRDial dashboard offers users a range of functionalities, including access to an inbuilt chatbot for non-simulator tasks and the ability to manage patient personas.

CoRDial in Action

Personalisation

Normal Vision

This video demonstrates CoRDial's conversational ability for users without visual impairments. The Legent simulator was used in this video to demonstrate the robot's actions

Color Blindness Adaptation

This video demonstrates CoRDial's conversational ability for users with visual impairments. The Legent simulator was used in this video to demonstrate the robot's actions

Funding

EMERGENCE EPSRC Logo

EMERGENCE EPSRC Healthcare Technologies Network+ under the Robotics for Frailty Challenge

RGU Catalyst Fund Logo

The Catalyst Fund at Robert Gordon University (RGU)

CoRDial Team

NW

Prof Nirmalie Wiratunga

Principal Investigator

KC

Prof Kay Cooper

Team Member

DC

Dr. David Corsar

Team Member

IN

Dr. Ikechukwu Nkisi-Orji

Team Member

VW

Vihanga Wijayasekara

Team Member

PS

Pedram Salimi

Team Member

EH

Erin Hart-Winks

Team Member

EC

Elsa Cox

Team Member

LJ

Lasal Jayawardana

Team Member