Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification

Simple item page
dc.article.number1081
dc.catalogadoraba
dc.contributor.authorWu, Derek
dc.contributor.authorSmith, Delaney
dc.contributor.authorVanBerlo, Blake
dc.contributor.authorRoshankar, Amir
dc.contributor.authorLee, Hoseok
dc.contributor.authorLi, Brian
dc.contributor.authorAli, Faraz
dc.contributor.authorRahman, Marwan
dc.contributor.authorBasmaji, John
dc.contributor.authorTschirhart, Jared
dc.contributor.authorFord, Alex
dc.contributor.authorVanBerlo, Bennett
dc.contributor.authorDurvasula, Ashritha
dc.contributor.authorVannelli, Claire
dc.contributor.authorDave, Chintan
dc.contributor.authorDeglint, Jason
dc.contributor.authorHo, Jordan
dc.contributor.authorChaudhary, Rushil
dc.contributor.authorClausdorff Fiedler, Hans Jurgen
dc.contributor.authorPrager, Ross
dc.contributor.authorMillington, Scott
dc.contributor.authorShah, Samveg
dc.contributor.authorBuchanan, Brian
dc.contributor.authorArntfield, Robert
dc.date.accessioned2024-06-06T18:07:01Z
dc.date.available2024-06-06T18:07:01Z
dc.date.issued2024
dc.description.abstractDeep learning (DL) models for medical image classification frequently struggle to generalize to data from outside institutions. Additional clinical data are also rarely collected to comprehensively assess and understand model performance amongst subgroups. Following the development of a single-center model to identify the lung sliding artifact on lung ultrasound (LUS), we pursued a validation strategy using external LUS data. As annotated LUS data are relatively scarce—compared to other medical imaging data—we adopted a novel technique to optimize the use of limited external data to improve model generalizability. Externally acquired LUS data from three tertiary care centers, totaling 641 clips from 238 patients, were used to assess the baseline generalizability of our lung sliding model. We then employed our novel Threshold-Aware Accumulative Fine-Tuning (TAAFT) method to fine-tune the baseline model and determine the minimum amount of data required to achieve predefined performance goals. A subgroup analysis was also performed and Grad-CAM++ explanations were examined. The final model was fine-tuned on one-third of the external dataset to achieve 0.917 sensitivity, 0.817 specificity, and 0.920 area under the receiver operator characteristic curve (AUC) on the external validation dataset, exceeding our predefined performance goals. Subgroup analyses identified LUS characteristics that most greatly challenged the model’s performance. Grad-CAM++ saliency maps highlighted clinically relevant regions on M-mode images. We report a multicenter study that exploits limited available external data to improve the generalizability and performance of our lung sliding model while identifying poorly performing subgroups to inform future iterative improvements. This approach may contribute to efficiencies for DL researchers working with smaller quantities of external validation data.
dc.fechaingreso.objetodigital2024-06-06
dc.fuente.origenORCID
dc.identifier.doi10.3390/diagnostics14111081
dc.identifier.eissn2075-4418
dc.identifier.urihttps://www.mdpi.com/2075-4418/14/11/1081
dc.identifier.urihttps://repositorio.uc.cl/handle/11534/86555
dc.information.autorucEscuela de Medicina; Clausdorff Fiedler, Hans Jurgen; 0000-0002-0571-7815; 172140
dc.issue.numero11
dc.language.isoen
dc.nota.accesocontenido completo
dc.pagina.final15
dc.pagina.inicio1
dc.revistaDiagnostics
dc.rightsacceso abierto
dc.rights.licenseATTRIBUTION 4.0 INTERNATIONAL Deed
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectArtificial intelligence
dc.subjectDeep learning
dc.subjectExplainability
dc.subjectGeneralizability
dc.subjectLung ultrasound
dc.subjectLung sliding
dc.subjectMulticenter
dc.subjectPneumothorax
dc.subjectPOCUS
dc.subjectUltrasound
dc.subject.ddc610
dc.subject.deweyMedicina y saludes_ES
dc.titleImproving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification
dc.typeartículo
dc.volumen14
sipa.codpersvinculados172140
sipa.trazabilidadORCID;2024-05-27
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
diagnostics-14-01081-v2.pdf
Size:
2.89 MB
Format:
Adobe Portable Document Format
Description:
Download
Collections
Artículos de revistas