1
Ministry of communication and information technology, Egypt ; Thebes Higher Institute for Computer and Administrative Sciences, Egypt
(Ahmedsleem8000@gmail.com)
2
Faculty of Computers and Informatics, Zagazig University, Zagazig, Sharqiyah, 44519, Egypt
( ielhenawy@zu.edu.eg)
Abstract :
The Internet of Medical Things (IoMT) offers numerous advantages in the diagnosis, monitoring, and treatment of a wide variety of illnesses for both patients. COVID-19 has caused a global pandemic and turned out to be the utmost crucial danger threatening the whole world. Thus, scholars’ attention moved toward Deep learning (DL) and IoMT for developing automated systems for COVID-19 diagnosis and/or prognosis based on chest computed tomography (CT) scans, and it has shown great success in several tasks, including classification and segmentation. Nevertheless, developing and training a superior DL approach necessitates accumulating a substantial amount of patients’ CT scans together with their labels. This is an expensive and time-consuming task that restricts attaining large enough data from a single site/institution, However, owing to the necessity for protecting data privacy, it is difficult to accumulate the data from several sites and store them at a centralized server. Federated learning (FL) alleviates the need for centralized data by spreading the public segmentation model to different institutional models, training the segmentation model at the institution, and followingly calculating the mean of the parameters in the public model. Nevertheless, researchers advocated that private information could be restored using the parameters of the model. This study presents a privacy-protection technique for the challenge of multi-site COVID-19 segmentation. To tackle the challenge, we introduce the FL technique, in which a distributed optimization procedure is developed, and randomization techniques are proposed to change the joint parameters of private institutional segmentation models. Bearing in mind the complete heterogeneity of COVID-19 distributions from diverse institutions, we develop two domain adaptation (DA) techniques in the proposed FL design. We explore several applied characteristics of optimizing the FL approach and analyze the FL approach in comparison with alternate training approaches. Finally, the results validate that it is auspicious to employ multi-site non-shared CT scans to improve the COVID-19 infection segmentation.
Keywords :
Deep Learning; COVID-19 Diagnosis; Segmentation; Multi-site Data; Federated Learning; Domain.
References :
[1] C. Wang, P. W. Horby, F. G. Hayden, and G. F. Gao, “A novel coronavirus outbreak of global health concern,” The Lancet,
vol. 395, no. 10223, pp. 470–473, Feb 2020.
[2] C. Huang, Y. Wang et al., “Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China,” The Lancet,
vol. 395, no. 10223, pp. 497–506, Feb 2020.
[3] T. Ai, Z. Yang et al., “Correlation of chest CT and rt-pcr testing in coronavirus disease 2019 (COVID-19) in China: A report
of 1014 cases,” Radiology, vol. 2019, p. 200642, Feb 2020.
[4] X. Ouyang, J. Huo, L. Xia, F. Shan, J. Liu, Z. Mo, et al., "Dual-Sampling Attention Network for Diagnosis of COVID-19
from Community Acquired Pneumonia," IEEE Transactions on Medical Imaging, 2020.
[5] H. Kang, L. Xia, F. Yan, Z. Wan, F. Shi, H. Yuan, et al., "Diagnosis of coronavirus disease 2019 (covid-19) with structured
latent multi-view representation learning," IEEE transactions on medical imaging, 2020.
[6] X. Wang, X. Deng, Q. Fu, Q. Zhou, J. Feng, H. Ma, et al., "A Weakly-supervised Framework for COVID-19 Classification
and Lesion Localization from Chest CT," IEEE Transactions on Medical Imaging, 2020.
[7] J. Wang, Y. Bao, Y. Wen, H. Lu, H. Luo, Y. Xiang , et al., "Prior-Attention Residual Learning for More Discriminative
COVID-19 Screening in CT Images," IEEE Transactions on Medical Imaging, 2020.
[8] Z. Han, B. Wei, Y. Hong, T. Li, J. Cong, X. Zhu, et al., "Accurate Screening of COVID-19 using Attention Based Deep 3D
Multiple Instance Learning," IEEE Transactions on Medical Imaging, 2020.
[9] D.-P. Fan, T. Zhou, G.-P. Ji, Y. Zhou, G. Chen, H. Fu, et al., "Inf-Net: Automatic COVID-19 Lung Infection Segmentation
from CT Images," IEEE Transactions on Medical Imaging, 2020.
[10] W. Xie, C. Jacobs, J.-P. Charbonnier, and B. van Ginneken, "Relational modeling for robust and efficient pulmonary lobe
segmentation in ct scans," IEEE transactions on medical imaging, 2020.
[11] Abdel-Basset, M., Chang, V., Hawash, H., Chakrabortty, R.K. and Ryan, M., 2021. FSS-2019-nCov: A deep learning
architecture for semi-supervised few-shot segmentation of COVID-19 infection. Knowledge-Based Systems, 212, p.106647.
[12] L. Zhou, Z. Li, J. Zhou, H. Li, Y. Chen, Y. Huang, et al., "A Rapid, Accurate and Machine-agnostic Segmentation and
Quantification Method for CT-based COVID-19 Diagnosis," IEEE Transactions on Medical Imaging, 2020.
[13] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, "Federated learning: Challenges, methods, and future directions," IEEE Signal
Processing Magazine, vol. 37, pp. 50-60, 2020.
[14] Xiaoxiao Li, Yufeng Gu, Nicha Dvornek, Lawrence H. Staib, Pamela Ventola, James S. Duncan, “Multi-site fMRI analysis
using privacy-preserving federated learning and domain adaptation: ABIDE results,” Medical Image Analysis, vol. 65, 2020.
[15] Q. Yang, Y. Liu, T. Chen, and Y. Tong, "Federated machine learning: Concept and applications," ACM Transactions on
Intelligent Systems and Technology (TIST), vol. 10, pp. 1-19, 2019.
[16] N. Lewis, H. Gazula, S. M. Plis, and V. D. Calhoun, "Decentralized distribution-sampled classification models with
application to brain imaging," Journal of neuroscience methods, vol. 329, p. 108418, 2020.
[17] Guan, H., Liu, Y., Yang, E., Yap, P.T., Shen, D. and Liu, M., 2021. Multi-site MRI harmonization via attention-guided deep
domain adaptation for brain disorder identification. Medical Image Analysis, 71, p.102076.
[18] W. Li, F. Milletarì, D. Xu, N. Rieke, J. Hancox, W. Zhu, et al., "Privacy-preserving federated brain tumour segmentation," in
International Workshop on Machine Learning in Medical Imaging , 2019, pp. 133-141.
[19] E. Ahn, A. Kumar, M. Fulham, D. Feng, and J. Kim, "Unsupervised Domain Adaptation to Classify Medical Images using
Zero-bias Convolutional Auto-encoders and Context-based Feature Augmentation," IEEE Transactions on Medical Imaging,
2020.
[20] Y. Zhang et al., "Collaborative Unsupervised Domain Adaptation for Medical Image Diagnosis," in IEEE Transactions on
Image Processing, vol. 29, pp. 7834-7844, 2020, doi: 10.1109/TIP.2020.3006377.
[21] S. Zhao, B. Li, X. Yue, Y. Gu, P. Xu, R. Hu, et al., "Multi-source domain adaptation for semantic segmentation," in Advances
in Neural Information Processing Systems, 2019, pp. 7287-7300.
[22] L. Song, C. Ma, G. Zhang, and Y. Zhang, "Privacy-Preserving Unsupervised Domain Adaptation in Federated Setting," IEEE
Access, vol. 8, pp. 143233-143240, 2020.
[23] P. C. M. Arachchige, P. Bertok, I. Khalil, D. Liu, S. Camtepe, and M. Atiquzzaman, "Local differential privacy for deep
learning," IEEE Internet of Things Journal, 2019.
[24] K. Chaudhuri, J. Imola, and A. Machanavajjhala, "Capacity bounded differential privacy," in Advances in Neural Information
Processing Systems, 2019, pp. 3474-3483.
[25] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in MICCAI.
Springer, 2015, pp. 234–241.
[26] D. J. Shah, "Multi-source domain adaptation with mixture of experts," Massachusetts Institute of Technology, 2019.
[27] D. Peterson, P. Kanani, and V. J. Marathe, "Private federated learning with domain adaptation," arXiv preprint
arXiv:1912.06733, 2019.
[28] J. Ma, Y. Wang, X. An, C. Ge, Z. Yu, J. Chen, et al., "Towards Efficient COVID-19 CT Annotation: A Benchmark for Lung
and Infection Segmentation," arXiv preprint arXiv:2004.12537, 2020.
[29] S. Morozov, A. Andreychenko, N. Pavlov, A. Vladzymyrskyy, N. Ledikhova, V. Gombolevskiy, et al., "MosMedData: Chest
CT Scans With COVID-19 Related Findings Dataset," arXiv preprint arXiv:2005.06465, 2020.
[30] M. de la Iglesia Vayá, J. M. Saborit, J. A. Montell, A. Pertusa, A. Bustos, M. Cazorla, et al., "BIMCV COVID-19+: a large
annotated dataset of RX and CT images from COVID-19 patients," arXiv preprint arXiv:2006.01174, 2020.
[31] Y. Tai, B. Gao, Q. Li, Z. Yu, C. Zhu and V. Chang, "Trustworthy and Intelligent COVID-19 Diagnostic IoMT Through XR
and Deep-Learning-Based Clinic Data Access," in IEEE Internet of Things Journal, vol. 8, no. 21, pp. 15965-15976, 1 Nov.1,
2021, doi: 10.1109/JIOT.2021.3055804.
[32] A. Rahman, M. S. Hossain, N. A. Alrajeh and F. Alsolami, "Adversarial Examples—Security Threats to COVID-19 Deep
Learning Systems in Medical IoT Devices," in IEEE Internet of Things Journal, vol. 8, no. 12, pp. 9603-9610, 15 June15,
2021, doi: 10.1109/JIOT.2020.3013710.
[33] T. Kitrungrotsakul et al., "Attention-RefNet: Interactive Attention Refinement Network for Infected Area Segmentation of
COVID-19," in IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 7, pp. 2363 -2373, July 2021, doi:
10.1109/JBHI.2021.3082527.
[34] Y. Zhang, Q. Liao, L. Yuan, H. Zhu, J. Xing and J. Zhang, "Exploiting Shared Knowledge From Non-COVID Lesions for
Annotation-Efficient COVID-19 CT Lung Infection Segmentation," in IEEE Journal of Biomedical and Health Informatics,
vol. 25, no. 11, pp. 4152-4162, Nov. 2021, doi: 10.1109/JBHI.2021.3106341.
[35] C. Li et al., "Self-Ensembling Co-Training Framework for Semi-Supervised COVID-19 CT Segmentation," in IEEE Journal
of Biomedical and Health Informatics, vol. 25, no. 11, pp. 4140-4151, Nov. 2021, doi: 10.1109/JBHI.2021.3103646.
[36] R. Wang, C. Ji, Y. Zhang and Y. Li, "Focus, Fusion, and Rectify: Context-Aware Learning for COVID-19 Lung Infection
Segmentation," in IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 1, pp. 12 -24, Jan. 2022, doi:
10.1109/TNNLS.2021.3126305.
[37] S. Yang et al., "Learning COVID-19 Pneumonia Lesion Segmentation From Imperfect Annotations via Divergence-Aware
Selective Training," in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 8, pp. 3673 -3684, Aug. 2022, doi:
10.1109/JBHI.2022.3172978.
[38] H. Zhao et al., "SC2Net: A Novel Segmentation-Based Classification Network for Detection of COVID-19 in Chest X-Ray
Images," in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 8, pp. 4032 -4043, Aug. 2022, doi:
10.1109/JBHI.2022.3177854.
[39] L. Song, C. Ma, G. Zhang and Y. Zhang, "Privacy-Preserving Unsupervised Domain Adaptation in Federated Setting," in
IEEE Access, vol. 8, pp. 143233-143240, 2020, doi: 10.1109/ACCESS.2020.3014264.
[40] Y. Kang, Y. He, J. Luo, T. Fan, Y. Liu and Q. Yang, "Privacy-preserving Federated Adversarial Domain Adaptation over
Feature Groups for Interpretability," in IEEE Transactions on Big Data, 2022, doi: 10.1109/TBDATA.2022.3188292.
[41] J. Liang, D. Hu, Y. Wang, R. He and J. Feng, "Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis
Transfer and Labeling Transfer," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp.
8602-8617, 1 Nov. 2022, doi: 10.1109/TPAMI.2021.3103390.
[42] W. Ding, M. Abdel-Basset, H. Hawash and O. M. Elkomy, "MT-nCov-Net: A Multitask Deep-Learning Framework for
Efficient Diagnosis of COVID-19 Using Tomography Scans," in IEEE Transactions on Cybernetics, doi:
10.1109/TCYB.2021.3123173.
[43] M. Abdel-Basset, H. Hawash and V. Chang, "FV-Seg-Net: Fully Volumetric Network for Accurate Segmentation of COVID-19 Lesions from Chest CT Scans," in IEEE Transactions on Industrial Informatics, doi: 10.1109/TII.2022.3146175.
[44] Ding, W., Abdel-Basset, M. and Hawash, H., 2021. RCTE: A reliable and consistent temporal-ensembling framework for
semi-supervised segmentation of COVID-19 lesions. Information sciences, 578, pp.559-573.
[45] C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating Noise to Sensitivity in Private Data Analysis,” J. Priv.
Confidentiality, 2017, doi: 10.29012/jpc.v7i3.405.
| Style | # |
|---|---|
| MLA | Ahmed Sleem, Ibrahim Elhenawy. "Collaborative Segmentation of COVID-19 From non-IID Topographies in the Internet of Medical Things (IoMT)." Journal of Intelligent Systems and Internet of Things, Vol. 7, No. 2, 2022 ,PP. 08-21 (Doi : https://doi.org/10.54216/JISIoT.070201) |
| APA | Ahmed Sleem, Ibrahim Elhenawy. (2022). Collaborative Segmentation of COVID-19 From non-IID Topographies in the Internet of Medical Things (IoMT). Journal of Journal of Intelligent Systems and Internet of Things, 7 ( 2 ), 08-21 (Doi : https://doi.org/10.54216/JISIoT.070201) |
| Chicago | Ahmed Sleem, Ibrahim Elhenawy. "Collaborative Segmentation of COVID-19 From non-IID Topographies in the Internet of Medical Things (IoMT)." Journal of Journal of Intelligent Systems and Internet of Things, 7 no. 2 (2022): 08-21 (Doi : https://doi.org/10.54216/JISIoT.070201) |
| Harvard | Ahmed Sleem, Ibrahim Elhenawy. (2022). Collaborative Segmentation of COVID-19 From non-IID Topographies in the Internet of Medical Things (IoMT). Journal of Journal of Intelligent Systems and Internet of Things, 7 ( 2 ), 08-21 (Doi : https://doi.org/10.54216/JISIoT.070201) |
| Vancouver | Ahmed Sleem, Ibrahim Elhenawy. Collaborative Segmentation of COVID-19 From non-IID Topographies in the Internet of Medical Things (IoMT). Journal of Journal of Intelligent Systems and Internet of Things, (2022); 7 ( 2 ): 08-21 (Doi : https://doi.org/10.54216/JISIoT.070201) |
| IEEE | Ahmed Sleem, Ibrahim Elhenawy, Collaborative Segmentation of COVID-19 From non-IID Topographies in the Internet of Medical Things (IoMT), Journal of Journal of Intelligent Systems and Internet of Things, Vol. 7 , No. 2 , (2022) : 08-21 (Doi : https://doi.org/10.54216/JISIoT.070201) |