Reinforcement learning for active modality selection during diagnosis - Physics, Radiobiology, Medical Imaging, and Simulation (PRIMES) Access content directly
Conference Papers Year :

Reinforcement learning for active modality selection during diagnosis


Diagnosis through imaging generally requires the combination of several modalities. Algorithms for data fusion allow merging information from different sources, mostly combining all images in a single step. In contrast, much less attention has been given to the incremental addition of new data descriptors, and the consideration of their costs (which can cover economic costs but also patient comfort and safety). In this work, we formalise clinical diagnosis of a patient as a sequential process of decisions, each of these decisions being whether to take an additional acquisition, or, if there is enough information, to end the examination and produce a diagnosis. We formulate the goodness of a diagnosis process as a combination of the classification accuracy minus the cost of the acquired modalities. To obtain a policy, we apply reinforcement learning, which recommends the next modality to incorporate based on data acquired at previous stages and aiming at maximising the accuracy/cost trade-off. This policy therefore performs medical diagnosis and patient-wise feature selection simultaneously. We demonstrate the relevance of this strategy on two binary classification datasets: a subset of a public heart disease database, including 531 instances with 11 scalar features, and a private echocardiographic dataset including signals from 5 standard image sequences used to assess cardiac function (2 speckle tracking, 2 flow Doppler and tissue Doppler), from 188 patients suffering hypertension, and 60 controls. For each individual, our algorithm allows acquiring only the modalities relevant for the diagnosis, avoiding low-information acquisitions, which both resulted in higher stability of the chosen modalities and better classification performance under a limited budget.
Fichier principal
Vignette du fichier
MICCAI_2022___RL.pdf (718.92 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03752018 , version 1 (16-08-2022)


  • HAL Id : hal-03752018 , version 1


Gabriel Bernardino, Anders Jonsson, Filip Loncaric, Pablo-Miki Marti Castellote, Marta Sitges, et al.. Reinforcement learning for active modality selection during diagnosis. 25th International Conference on Medical Image Computing and Computer Assisted Intervention, Sep 2022, Singapore, Singapore. ⟨hal-03752018⟩
87 View
106 Download


Gmail Facebook Twitter LinkedIn More