A Multimodal Fusion Framework for Moyamoya Disease Diagnosis, Severity Staging, and Evidence-Grounded Report Generation
编号:136
访问权限:仅限参会人
更新:2026-11-10 16:52:23 浏览:8次
Online
摘要
Moyamoya disease (MMD) is a rare, progressive cerebrovascular steno-occlusive disorder requiring angiographic confirmation and Suzuki staging. We present the Omni-Agent Cerebrovascular framework, an end-to-end multimodal system that integrates retinal fundus images, magnetic resonance angiography (MRA) volumes, and clinical variables through gated cross-modal attention fusion for simultaneous MMD diagnosis, severity staging, and evidence-grounded report generation. Three perception branches—ResNeXt-50 for bilateral fundus images, 3D ResNet-18 for MRA, and an MLP for clinical data—produce modality-specific embeddings that are fused via learned gating weights. The fused representation feeds a binary MMD classifier and an ordinal Suzuki staging head using cumulative probabilities. An instruction-tuned Qwen2.5-VL-7B-Instruct model with Low-Rank Adaptation (LoRA) generates evidence-grounded clinical reports strictly from structured upstream predictions. Evaluated on 263 patients (127 MMD, 136 controls), the full multimodal system achieved an AUROC of 0.961 (95% CI: 0.934–0.988) for MMD detection, a weighted Cohen's κ of 0.741 for Suzuki staging, and a report generation hallucination rate of 3.8% with 91.3% clinical factuality.
关键词
Moyamoya disease,multimodal fusion,gated cross-modal attention,ordinal regression,medical report generation,retinal fundus imaging,deep learning
稿件作者
ANIRUDH M R
Karunya Institute of Technology and Sciences
Shirley C P
Karunya Institute of Technology and Sciences
发表评论