WithinMyVoice: Training system for Emotion-Enabled Synthetic Voice Augmentation
Catarina Santos
ABSTRACT
People living with Motor Neuron Disease (plwMND) often rely on Augmentative and Alternative Communication (AAC) devices to communicate as the disease progresses. Voice banking creates Personalised Synthetic Voice (PSV), however, the latter still lacks emotional expressiveness. This paper introduces a preliminary prototype of ‘WithinMyVoice’, a novel PSV training system, integrating voice banking with emotional and contextual sensitivity, delivered by a software application featuring digital humans, conversational artificial intelligence systems while capturing voice, facial expression and biosignals for emotion recognition. The double diamond design framework was used. Qualitative analysis was conducted on extracts from eight user stories and one interview and revealed that current PSV can be monotone and missing to express real-time emotions; voice banking can have a positive psychological effect on plwMND; finally, an in-ear wearable would be acceptable to support multimodal emotion recognition. ‘WithinMyVoice’ transforms the data collection experience from a purely data-driven task to human-centred conversational experience with two main outcomes: potential psychological support during training, and collect data that will enable the generation of PSV infused with emotion to enrich plwMND’s social interactions.