2690-6791ISSN (Online)
2769-786XISSN (Print)
Volume 7 , Issue 2 , PP: 08-21, 2022 | Cite this article as | XML | Html | PDF | Full Length Article
Ahmed Sleem 1 * , Ibrahim Elhenawy 2
Doi: https://doi.org/10.54216/JISIoT.070201
The Internet of Medical Things (IoMT) offers numerous advantages in the diagnosis, monitoring, and treatment of a wide variety of illnesses for both patients. COVID-19 has caused a global pandemic and turned out to be the utmost crucial danger threatening the whole world. Thus, scholars’ attention moved toward Deep learning (DL) and IoMT for developing automated systems for COVID-19 diagnosis and/or prognosis based on chest computed tomography (CT) scans, and it has shown great success in several tasks, including classification and segmentation. Nevertheless, developing and training a superior DL approach necessitates accumulating a substantial amount of patients’ CT scans together with their labels. This is an expensive and time-consuming task that restricts attaining large enough data from a single site/institution, However, owing to the necessity for protecting data privacy, it is difficult to accumulate the data from several sites and store them at a centralized server. Federated learning (FL) alleviates the need for centralized data by spreading the public segmentation model to different institutional models, training the segmentation model at the institution, and followingly calculating the mean of the parameters in the public model. Nevertheless, researchers advocated that private information could be restored using the parameters of the model. This study presents a privacy-protection technique for the challenge of multi-site COVID-19 segmentation. To tackle the challenge, we introduce the FL technique, in which a distributed optimization procedure is developed, and randomization techniques are proposed to change the joint parameters of private institutional segmentation models. Bearing in mind the complete heterogeneity of COVID-19 distributions from diverse institutions, we develop two domain adaptation (DA) techniques in the proposed FL design. We explore several applied characteristics of optimizing the FL approach and analyze the FL approach in comparison with alternate training approaches. Finally, the results validate that it is auspicious to employ multi-site non-shared CT scans to improve the COVID-19 infection segmentation.
Deep Learning , COVID-19 Diagnosis , Segmentation , Multi-site Data , Federated Learning , Domain.
[1] C. Wang, P. W. Horby, F. G. Hayden, and G. F. Gao, “A novel coronavirus outbreak of global health concern,” The Lancet, vol. 395, no. 10223, pp. 470–473, Feb 2020.
[2] C. Huang, Y. Wang et al., “Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China,” The Lancet, vol. 395, no. 10223, pp. 497–506, Feb 2020.
[3] T. Ai, Z. Yang et al., “Correlation of chest CT and rt-pcr testing in coronavirus disease 2019 (COVID-19) in China: A report of 1014 cases,” Radiology, vol. 2019, p. 200642, Feb 2020.
[4] X. Ouyang, J. Huo, L. Xia, F. Shan, J. Liu, Z. Mo, et al., "Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumonia," IEEE Transactions on Medical Imaging, 2020.
[5] H. Kang, L. Xia, F. Yan, Z. Wan, F. Shi, H. Yuan, et al., "Diagnosis of coronavirus disease 2019 (covid-19) with structured latent multi-view representation learning," IEEE transactions on medical imaging, 2020.
[6] X. Wang, X. Deng, Q. Fu, Q. Zhou, J. Feng, H. Ma, et al., "A Weakly-supervised Framework for COVID-19 Classification and Lesion Localization from Chest CT," IEEE Transactions on Medical Imaging, 2020.
[7] J. Wang, Y. Bao, Y. Wen, H. Lu, H. Luo, Y. Xiang , et al., "Prior-Attention Residual Learning for More Discriminative COVID-19 Screening in CT Images," IEEE Transactions on Medical Imaging, 2020.
[8] Z. Han, B. Wei, Y. Hong, T. Li, J. Cong, X. Zhu, et al., "Accurate Screening of COVID-19 using Attention Based Deep 3D Multiple Instance Learning," IEEE Transactions on Medical Imaging, 2020.
[9] D.-P. Fan, T. Zhou, G.-P. Ji, Y. Zhou, G. Chen, H. Fu, et al., "Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images," IEEE Transactions on Medical Imaging, 2020.
[10] W. Xie, C. Jacobs, J.-P. Charbonnier, and B. van Ginneken, "Relational modeling for robust and efficient pulmonary lobe segmentation in ct scans," IEEE transactions on medical imaging, 2020.
[11] Abdel-Basset, M., Chang, V., Hawash, H., Chakrabortty, R.K. and Ryan, M., 2021. FSS-2019-nCov: A deep learning architecture for semi-supervised few-shot segmentation of COVID-19 infection. Knowledge-Based Systems, 212, p.106647.
[12] L. Zhou, Z. Li, J. Zhou, H. Li, Y. Chen, Y. Huang, et al., "A Rapid, Accurate and Machine-agnostic Segmentation and Quantification Method for CT-based COVID-19 Diagnosis," IEEE Transactions on Medical Imaging, 2020.
[13] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, "Federated learning: Challenges, methods, and future directions," IEEE Signal Processing Magazine, vol. 37, pp. 50-60, 2020.
[14] Xiaoxiao Li, Yufeng Gu, Nicha Dvornek, Lawrence H. Staib, Pamela Ventola, James S. Duncan, “Multi-site fMRI analysis using privacy-preserving federated learning and domain adaptation: ABIDE results,” Medical Image Analysis, vol. 65, 2020.
[15] Q. Yang, Y. Liu, T. Chen, and Y. Tong, "Federated machine learning: Concept and applications," ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, pp. 1-19, 2019.
[16] N. Lewis, H. Gazula, S. M. Plis, and V. D. Calhoun, "Decentralized distribution-sampled classification models with application to brain imaging," Journal of neuroscience methods, vol. 329, p. 108418, 2020.
[17] Guan, H., Liu, Y., Yang, E., Yap, P.T., Shen, D. and Liu, M., 2021. Multi-site MRI harmonization via attention-guided deep domain adaptation for brain disorder identification. Medical Image Analysis, 71, p.102076.
[18] W. Li, F. Milletarì, D. Xu, N. Rieke, J. Hancox, W. Zhu, et al., "Privacy-preserving federated brain tumour segmentation," in International Workshop on Machine Learning in Medical Imaging , 2019, pp. 133-141.
[19] E. Ahn, A. Kumar, M. Fulham, D. Feng, and J. Kim, "Unsupervised Domain Adaptation to Classify Medical Images using Zero-bias Convolutional Auto-encoders and Context-based Feature Augmentation," IEEE Transactions on Medical Imaging, 2020.
[20] Y. Zhang et al., "Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis," in IEEE Transactions on Image Processing, vol. 29, pp. 7834-7844, 2020, doi: 10.1109/TIP.2020.3006377.
[21] S. Zhao, B. Li, X. Yue, Y. Gu, P. Xu, R. Hu, et al., "Multi-source domain adaptation for semantic segmentation," in Advances in Neural Information Processing Systems, 2019, pp. 7287-7300.
[22] L. Song, C. Ma, G. Zhang, and Y. Zhang, "Privacy-Preserving Unsupervised Domain Adaptation in Federated Setting," IEEE Access, vol. 8, pp. 143233-143240, 2020.
[23] P. C. M. Arachchige, P. Bertok, I. Khalil, D. Liu, S. Camtepe, and M. Atiquzzaman, "Local differential privacy for deep learning," IEEE Internet of Things Journal, 2019.
[24] K. Chaudhuri, J. Imola, and A. Machanavajjhala, "Capacity bounded differential privacy," in Advances in Neural Information Processing Systems, 2019, pp. 3474-3483.
[25] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in MICCAI. Springer, 2015, pp. 234–241.
[26] D. J. Shah, "Multi-source domain adaptation with mixture of experts," Massachusetts Institute of Technology, 2019.
[27] D. Peterson, P. Kanani, and V. J. Marathe, "Private federated learning with domain adaptation," arXiv preprint arXiv:1912.06733, 2019.
[28] J. Ma, Y. Wang, X. An, C. Ge, Z. Yu, J. Chen, et al., "Towards Efficient COVID-19 CT Annotation: A Benchmark for Lung and Infection Segmentation," arXiv preprint arXiv:2004.12537, 2020.
[29] S. Morozov, A. Andreychenko, N. Pavlov, A. Vladzymyrskyy, N. Ledikhova, V. Gombolevskiy, et al., "MosMedData: Chest CT Scans With COVID-19 Related Findings Dataset," arXiv preprint arXiv:2005.06465, 2020.
[30] M. de la Iglesia Vayá, J. M. Saborit, J. A. Montell, A. Pertusa, A. Bustos, M. Cazorla, et al., "BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patients," arXiv preprint arXiv:2006.01174, 2020.
[31] Y. Tai, B. Gao, Q. Li, Z. Yu, C. Zhu and V. Chang, "Trustworthy and Intelligent COVID-19 Diagnostic IoMT Through XR and Deep-Learning-Based Clinic Data Access," in IEEE Internet of Things Journal, vol. 8, no. 21, pp. 15965-15976, 1 Nov.1, 2021, doi: 10.1109/JIOT.2021.3055804.
[32] A. Rahman, M. S. Hossain, N. A. Alrajeh and F. Alsolami, "Adversarial Examples—Security Threats to COVID-19 Deep Learning Systems in Medical IoT Devices," in IEEE Internet of Things Journal, vol. 8, no. 12, pp. 9603-9610, 15 June15, 2021, doi: 10.1109/JIOT.2020.3013710.
[33] T. Kitrungrotsakul et al., "Attention-RefNet: Interactive Attention Refinement Network for Infected Area Segmentation of COVID-19," in IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 7, pp. 2363 -2373, July 2021, doi: 10.1109/JBHI.2021.3082527.
[34] Y. Zhang, Q. Liao, L. Yuan, H. Zhu, J. Xing and J. Zhang, "Exploiting Shared Knowledge From Non-COVID Lesions for Annotation-Efficient COVID-19 CT Lung Infection Segmentation," in IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 11, pp. 4152-4162, Nov. 2021, doi: 10.1109/JBHI.2021.3106341.
[35] C. Li et al., "Self-Ensembling Co-Training Framework for Semi-Supervised COVID-19 CT Segmentation," in IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 11, pp. 4140-4151, Nov. 2021, doi: 10.1109/JBHI.2021.3103646.
[36] R. Wang, C. Ji, Y. Zhang and Y. Li, "Focus, Fusion, and Rectify: Context-Aware Learning for COVID-19 Lung Infection
Segmentation," in IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 1, pp. 12 -24, Jan. 2022, doi:
10.1109/TNNLS.2021.3126305.
[37] S. Yang et al., "Learning COVID-19 Pneumonia Lesion Segmentation From Imperfect Annotations via Divergence-Aware Selective Training," in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 8, pp. 3673 -3684, Aug. 2022, doi: 10.1109/JBHI.2022.3172978.
[38] H. Zhao et al., "SC2Net: A Novel Segmentation-Based Classification Network for Detection of COVID-19 in Chest X-Ray Images," in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 8, pp. 4032 -4043, Aug. 2022, doi: 10.1109/JBHI.2022.3177854.
[39] L. Song, C. Ma, G. Zhang and Y. Zhang, "Privacy-Preserving Unsupervised Domain Adaptation in Federated Setting," in IEEE Access, vol. 8, pp. 143233-143240, 2020, doi: 10.1109/ACCESS.2020.3014264.
[40] Y. Kang, Y. He, J. Luo, T. Fan, Y. Liu and Q. Yang, "Privacy-preserving Federated Adversarial Domain Adaptation over Feature Groups for Interpretability," in IEEE Transactions on Big Data, 2022, doi: 10.1109/TBDATA.2022.3188292.
[41] J. Liang, D. Hu, Y. Wang, R. He and J. Feng, "Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 8602-8617, 1 Nov. 2022, doi: 10.1109/TPAMI.2021.3103390.
[42] W. Ding, M. Abdel-Basset, H. Hawash and O. M. Elkomy, "MT-nCov-Net: A Multitask Deep-Learning Framework for Efficient Diagnosis of COVID-19 Using Tomography Scans," in IEEE Transactions on Cybernetics, doi: 10.1109/TCYB.2021.3123173.
[43] M. Abdel-Basset, H. Hawash and V. Chang, "FV-Seg-Net: Fully Volumetric Network for Accurate Segmentation of COVID-19 Lesions from Chest CT Scans," in IEEE Transactions on Industrial Informatics, doi: 10.1109/TII.2022.3146175.
[44] Ding, W., Abdel-Basset, M. and Hawash, H., 2021. RCTE: A reliable and consistent temporal-ensembling framework forsemi-supervised segmentation of COVID-19 lesions. Information sciences, 578, pp.559-573.
[45] C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating Noise to Sensitivity in Private Data Analysis,” J. Priv. Confidentiality, 2017, doi: 10.29012/jpc.v7i3.405.
