Our #ACL2023NLP paper, “Are Experts Needed? On Human Evaluation of Counselling Reflection Generation” is now online!
Find it at: https://t.co/VUtakmtIIG
(1/6)
Proud to share that our paper "Are Experts Needed?
On Human Evaluation of Counselling Reflection
Generation'" has been accepted to the main
conference of ACL 2023 #ACL2023NLP! Huge thanks
to my co-authors @ZixiuAlexWu, @EhudReiter, @RimHelaoui, @diegoref, and Daniele
Riboni!
We're online!
"Creation, Analysis and Evaluation of AnnoMI, a Dataset of Expert-Annotated Counselling Dialogues", with @ZixiuAlexWu , @kunwar_vsrana, @RimHelaoui@diegoref and Daniele Riboni) has been published in the Future Internet journal! Find it at: https://t.co/o0FCZzbdqK
I'm proud to share that my PhD work on empathetic dialogue AI for healthcare has been recognised in the EU's Horizon Magazine (check the "Empathetic bots" section)! If you're interested, let's have a chat!
Chatbots that can display empathy, algorithms for home assistance helpers, and robots to help children learn – robotics & AI are poised to kick off big changes in our societies. Learn more ➡️ https://t.co/ifNgB8sXpf
#HorizonEU
We also propose a non-expert evaluation scheme based on laypeople's free-text descriptions of errors in generated reflections. We will be conducting a mass non-expert annotation effort using this scheme and releasing the data to the public.
Camera-ready: https://t.co/s7wNzrkaqm
Very excited to announce that I will be presenting our work (w/ @simoneballoccu@RimHelaoui@diegoref Daniele Riboni) "Towards In-Context Non-Expert Evaluation of Reflection Generation for Counselling Conversations" at GEM 2022 #GEM2022, co-located with EMNLP 2022 @emnlpmeeting!
Human evaluation is expensive for machine-generated therapist reflections because of the highly specialised domain, so we propose to disentangle it into 1) non-expert evaluation wrt coherence & consistency with context and 2) expert-evaluation wrt conformity to therapy guidelines
@YRRSDS_Official@HeriotWattUni Hi I’ve registered twice but I haven’t got any WhatsApp invite in my email inbox and there’s nothing in my spam folder either, so could you perhaps send me the link privately? Thank you!
@ISCAInterspeech Thank you for this! Quick question: when will on-site presenters be informed whether they will be giving poster or oral presentations?
It keeps happening that I go to events and meet researchers looking for annotated data in the psychological domain. In 2022 we (with @kunwar_vsrana & @ZixiuAlexWu) released AnnoMI exactly to face this problem (https://t.co/qqLkTFPIoW).
Here's why AnnoMI could be useful for you:
We really hope that our efforts will be beneficial for the research community, as this type of data is usually hard to access. Moreover, the annotation protocol has been designed with the aid of experts as well, to guarantee the highest possible quality in the resulting data!
We have made public the AnnoMI dataset at https://t.co/2G7bZLUuyO! It's the the first publicly & freely available dataset of expert-annotated therapy dialogues demonstrating high- & low-quality motivational interviewing (an important counselling technique). Please try it out :)
Our paper "Anno-MI: A Dataset of Expert-Annotated Counselling Dialogues" (w/ @simoneballoccu, @kunwar_vsrana, @RimHelaoui, @EhudReiter, @diegoref, Daniele Riboni) detailing the collection of this dataset has been accepted to ICASSP 2022. Paper available at https://t.co/K0KsVgA4NI
Happy to announce that our paper "Anno-MI: a dataset of expert-annotated counselling dialogues" (with @ZixiuAlexWu, @kunwar_vsrana, @EhudReiter , @diegoref, Rim Elahoui and Daniele Riboni) has been accepted at ISASSP 2022! (1/n)