Skip to Main content Skip to Navigation
New interface
Conference papers

Reinforcement learning for active modality selection during diagnosis

Abstract : Diagnosis through imaging generally requires the combination of several modalities. Algorithms for data fusion allow merging information from different sources, mostly combining all images in a single step. In contrast, much less attention has been given to the incremental addition of new data descriptors, and the consideration of their costs (which can cover economic costs but also patient comfort and safety). In this work, we formalise clinical diagnosis of a patient as a sequential process of decisions, each of these decisions being whether to take an additional acquisition, or, if there is enough information, to end the examination and produce a diagnosis. We formulate the goodness of a diagnosis process as a combination of the classification accuracy minus the cost of the acquired modalities. To obtain a policy, we apply reinforcement learning, which recommends the next modality to incorporate based on data acquired at previous stages and aiming at maximising the accuracy/cost trade-off. This policy therefore performs medical diagnosis and patient-wise feature selection simultaneously. We demonstrate the relevance of this strategy on two binary classification datasets: a subset of a public heart disease database, including 531 instances with 11 scalar features, and a private echocardiographic dataset including signals from 5 standard image sequences used to assess cardiac function (2 speckle tracking, 2 flow Doppler and tissue Doppler), from 188 patients suffering hypertension, and 60 controls. For each individual, our algorithm allows acquiring only the modalities relevant for the diagnosis, avoiding low-information acquisitions, which both resulted in higher stability of the chosen modalities and better classification performance under a limited budget.
Complete list of metadata
Contributor : Gabriel Bernardino Connect in order to contact the contributor
Submitted on : Tuesday, August 16, 2022 - 10:55:16 AM
Last modification on : Tuesday, October 18, 2022 - 4:31:03 AM
Long-term archiving on: : Thursday, November 17, 2022 - 6:19:47 PM


Files produced by the author(s)


  • HAL Id : hal-03752018, version 1


Gabriel Bernardino, Anders Jonsson, Filip Loncaric, Pablo-Miki Marti Castellote, Marta Sitges, et al.. Reinforcement learning for active modality selection during diagnosis. 25th International Conference on Medical Image Computing and Computer Assisted Intervention, Sep 2022, Singapore, Singapore. ⟨hal-03752018⟩



Record views


Files downloads