A Multimodal AI Platform for Mental Wellness Screening: Speech and Facial Emotion Analysis with Transparent Rule-Based Fusion and Crisis-Aware Conversational Assessment
编号:172 访问权限:仅限参会人 更新:2026-11-10 17:02:09 浏览:9次 Online

报告开始:2026年11月20日 15:15(Asia/Kolkata)

报告时间:15min

所在会场:[S4] Artificial Intelligence and Intelligent Computing [S4-3] Artificial Intelligence and Intelligent Computing

暂无文件

摘要
Multimodal mental wellness screening usually fuses speech, facial behaviour, lifestyle, and conversational signals with a trained neural component, even though no public dataset pairs these modalities with clinically validated wellness labels at meaningful scale. The MindWell platform instead combines a pretrained speech emotion recognizer, a pretrained facial emotion classifier paired with MediaPipederived geometric features, and a large language model (LLM) that conversationally administers the PHQ-9, fusing these outputs through a transparent, confidence-weighted rule-based layer in which every score traces to an explicit formula. A separate interpret-compute-phrase pipeline answers naturallanguage questions about a user's own history while guaranteeing that every number the LLM states was computed independently beforehand, a structural safeguard against hallucinated figures. The facial emotion classifier was benchmarked on the complete FER2013 test split (7,178 images), achieving 88.3% accuracy and 88.7% macro F1score, while the speech emotion model and other components remain limited to small-sample component-level verification. Additional evidence includes a real production incident in which the safety filter intercepted disallowed diagnostic language generated by the LLM, and concrete engineering failures encountered during development, including a published model whose documented architecture did not match its own stored weights. Where the data needed to justify a trained, opaque component does not exist, a transparent alternative is used instead, and results are reported at the scale actually measured rather than as a benchmark that scale cannot support.
关键词
Mental health screening, multimodal fusion, speech emotion recognition, facial emotion recognition, explainable AI, conversational AI, PHQ-9, large language models, crisis detection
报告人
Jerin Saji
Undergraduate Karunya Institute of Technology and Sciences

稿件作者
Jerin Saji Karunya Institute of Technology and Sciences
Divina T P Karunya Institute of Technology and Science
发表评论
验证码 看不清楚,更换一张
全部评论
重要日期
  • 会议日期

    11月20日

    2026

    至

    11月22日

    2026

  • 10月31日 2026

    初稿截稿日期

  • 11月07日 2026

    提前注册日期

  • 11月22日 2026

    注册截止日期

主办单位
United Societies of Science
承办单位
Pondicherry University
协办单位
IEEE Section
IEEE Madras Section
移动端
在手机上打开
小程序
打开微信小程序
客服
扫码或点此咨询