The research presents a comprehensive analysis of MLLMs specifically tailored for AIS self-management. It employs a framework consisting of various tasks to assess the models’ capabilities in visual question answering, domain knowledge, and patient education. Findings indicate a significant gap in performance, particularly in accurately detecting spinal deformities, signaling a crucial area for development.
Despite introducing enhancements like spinal keypoint prompting and a retrieval augmented generation (RAG) knowledge base, MLLMs still struggled with orientation understanding. The best accuracy achieved for detecting deformity locations was just 0.55, while direction accuracy was notably lower at 0.13. These limitations underscore the importance of continued research in this domain to enable personalized assistance in AIS care, revealing both risks in patient management and potential improvements through targeted AI advancements.
👉 Pročitaj original: arXiv AI Papers
