2692-4048ISSN (Online)
2770-0070ISSN (Print)
Volume 3 , Issue 1 , PP: 29-42, 2021 | Cite this article as | XML | Html | PDF | Full Length Article
Fairoz Q. Kareem 1 , Adnan Mohsin Abdulazeez 2
Doi: https://doi.org/10.54216/FPA.030102
With the development of technology and smart devices in the medical field, the computer system has become an essential part of this development to learn devices in the medical field. One of the learning methods is deep learning (DL), which is a branch of machine learning (ML). The deep learning approach has been used in this field because it is one of the modern methods of obtaining accurate results through its algorithms, and among these algorithms that are used in this field are convolutional neural networks (CNN) and recurrent neural networks (RNN). In this paper we reviewed what have researchers have done in their researches to solve fetal problems, then summarize and carefully discuss the applications in different tasks identified for segmentation and classification of ultrasound images. Finally, this study discussed the potential challenges and directions for applying deep learning in ultrasound image analysis.
Fetal Brain Deformities, Deep Learning Algorithms, Ultrasonic Images, Classification.
[1] N. S. Ahmed, "A fractal-based model to improve cooperation among physicians in distributed healthcare information systems," University of Malaya, 2013.
[2] O. Glenn and A. Barkovich, "Magnetic resonance imaging of the fetal brain and spine: an increasingly important tool in prenatal diagnosis, part 1," American Journal of Neuroradiology, vol. 27, pp. 1604-1611, 2006.
[3] D. Q. Zeebaree, A. M. Abdulazeez, D. A. Zebari, H. Haron and H. Nuzly, "Multi-level fusion in ultrasound for cancer detection based on uniform lbp features," Computers, Materials & Continua, vol. 66, no.3, pp. 3363–3382, 2021
[4] D. A. Zebari, D. Q. Zeebaree, A. M. Abdulazeez, H. Haron, and H. N. A. Hamed, "Improved Threshold Based and Trainable Fully Automated Segmentation for Breast Cancer Boundary and Pectoral Muscle in Mammogram Images," IEEE Access, vol. 8, pp. 203097-203116, 2020.
[5] Jahwar, A., & Ahmed, N. (2021). Swarm Intelligence Algorithms in Gene Selection Profile Based on Classification of Microarray Data: A Review. Journal of Applied Science and Technology Trends, 2(01), 01-09.
[6] Jahwar, A. F., & Abdulazeez, A. M. (2020). Meta-Heuristic Algorithms For K-Means Clustering: A Review. PalArch's Journal of Archaeology of Egypt/Egyptology, 17(7), 12002-12020.
[7] Zeebaree, D. Q., Haron, H., Abdulazeez, A. M., & Zebari, D. A. (2019, April). Trainable model based on new uniform LBP feature to identify the risk of the breast cancer. In 2019 International Conference on Advanced Science and Engineering (ICOASE) (pp. 106-111). IEEE.
[8] D. Paladini and P. Volpe, Ultrasound of congenital fetal anomalies: differential diagnosis and prognostic indicators: CRC Press, 2018.
[9] J. C. Hobbins, P. A. Grannum, R. L. Berkowitz, R. Silverman, and M. J. Mahoney, "Ultrasound in the diagnosis of congenital anomalies," American Journal of Obstetrics and Gynecology, vol. 134, pp. 331-345, 1979.
[10] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning vol. 1: MIT press Cambridge, 2016.
[11] D. Shen, G. Wu, and H.-I. Suk, "Deep learning in medical image analysis," Annual review of biomedical engineering, vol. 19, pp. 221-248, 2017.
[12] S. K. Zhou, H. Greenspan, and D. Shen, Deep learning for medical image analysis: Academic Press, 2017.
[13] N. Kussul, M. Lavreniuk, S. Skakun, and A. Shelestov, "Deep learning classification of land cover and crop types using remote sensing data," IEEE Geoscience and Remote Sensing Letters, vol. 14, pp. 778-782, 2017.
[14] D. Q. Zeebaree, H. Haron, A. M. Abdulazeez, and D. A. Zebari, "Machine learning and region growing for breast cancer segmentation," in 2019 International Conference on Advanced Science and Engineering (ICOASE), 2019, pp. 88-93.
[15] A. Holzinger, C. Biemann, C. S. Pattichis, and D. B. Kell, "What do we need to build explainable AI systems for the medical domain?," arXiv preprint arXiv:1712.09923, 2017.
[16] S. Han, H.-K. Kang, J.-Y. Jeong, M.-H. Park, W. Kim, W.-C. Bang, et al., "A deep learning framework for supporting the classification of breast lesions in ultrasound images," Physics in Medicine & Biology, vol. 62, p. 7714, 2017.
[17] A. A. Nair, T. D. Tran, A. Reiter, and M. A. L. Bell, "A deep learning based alternative to beamforming ultrasound images," in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 3359-3363.
[18] H. Li, J. Weng, Y. Shi, W. Gu, Y. Mao, Y. Wang, et al., "An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images," Scientific reports, vol. 8, pp. 1-12, 2018.
[19] Z. Liu, C. Yang, J. Huang, S. Liu, Y. Zhuo, and X. Lu, "Deep learning framework based on integration of S-Mask R-CNN and Inception-v3 for ultrasound image-aided diagnosis of prostate cancer," Future Generation Computer Systems, vol. 114, pp. 358-367, 2020.
[20] J. Zhu, W. Styler, and I. Calloway, "A CNN-based tool for automatic tongue contour tracking in ultrasound images," arXiv preprint arXiv:1907.10210, 2019.
[21] M. Miller, W. Nyborg, W. Dewey, M. Edwards, J. Abramowicz, and A. Brayman, "Hyperthermic teratogenicity, thermal dose and diagnostic ultrasound during pregnancy: implications of new standards on tissue heating," International journal of hyperthermia, vol. 18, pp. 361-384, 2002.
[22] T. L. Szabo, Diagnostic ultrasound imaging: inside out: Academic Press, 2004.
[23] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, pp. 84-90, 2017.
[24] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, 2015, pp. 234-241.
[25] A. Senior, V. Vanhoucke, P. Nguyen, and T. Sainath, "Deep neural networks for acoustic modeling in speech recognition," IEEE Signal processing magazine, 2012.
[26] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, et al., "Mastering the game of Go with deep neural networks and tree search," nature, vol. 529, pp. 484-489, 2016.
[27] H. Chen, D. Ni, J. Qin, S. Li, X. Yang, T. Wang, et al., "Standard plane localization in fetal ultrasound via domain transferred deep neural networks," IEEE journal of biomedical and health informatics, vol. 19, pp. 1627-1636, 2015.
[28] K. Wu, X. Chen, and M. Ding, "Deep learning based classification of focal liver lesions with contrast-enhanced ultrasound," Optik, vol. 125, pp. 4057-4063, 2014.
[29] L. Deng and D. Yu, "Deep learning: methods and applications," Foundations and trends in signal processing, vol. 7, pp. 197-387, 2014.
[30] C. Ouchicha, O. Ammor, and M. Meknassi, "CVDNet: A novel deep learning architecture for detection of coronavirus (Covid-19) from chest x-ray images," Chaos, Solitons & Fractals, vol. 140, p. 110245, 2020.
[31] D. Q. Zeebaree, H. Haron, A. M. Abdulazeez, and S. R. Zeebaree, "Combination of K-means clustering with Genetic Algorithm: A review," International Journal of Applied Engineering Research, vol. 12, pp. 14238-14245, 2017.
[32] S. Liu, Y. Wang, X. Yang, B. Lei, L. Liu, S. X. Li, et al., "Deep learning in medical ultrasound analysis: a review," Engineering, vol. 5, pp. 261-275, 2019.
[33] X. Yang, L. Yu, L. Wu, Y. Wang, D. Ni, J. Qin, et al., "Fine-grained recurrent neural networks for automatic prostate segmentation in ultrasound images," in Proceedings of the AAAI Conference on Artificial Intelligence, 2017.
[34] H. Chen, Q. Dou, D. Ni, J.-Z. Cheng, J. Qin, S. Li, et al., "Automatic fetal ultrasound standard plane detection using knowledge transferred recurrent neural networks," in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, pp. 507-514.
[35] F. Bargarai, A. Abdulazeez, V. Tiryaki, and D. Zeebaree, "Management of Wireless Communication Systems Using Artificial Intelligence-Based Software Defined Radio," 2020.
[36] Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE transactions on neural networks, vol. 5, pp. 157-166, 1994.
[37] A. Abdulazeez, B. Salim, D. Zeebaree, and D. Doghramachi, "Comparison of VPN Protocols at Network Layer Focusing on Wire Guard Protocol," 2020.
[38] H. Sak, A. W. Senior, and F. Beaufays, "Long short-term memory recurrent neural network architectures for large scale acoustic modeling," 2014.
[39] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, et al., "Learning phrase representations using RNN encoder-decoder for statistical machine translation," arXiv preprint arXiv:1406.1078, 2014.
[40] D. Q. Zeebaree, A. M. Abdulazeez, O. M. S. Hassan, D. A. Zebari, and J. N. Saeed, "Hiding Image by Using Contourlet Transform," ed: press, 2020.
[41] C. Yin, Y. Zhu, J. Fei, and X. He, "A deep learning approach for intrusion detection using recurrent neural networks," Ieee Access, vol. 5, pp. 21954-21961, 2017.
[42] D. Q. Zeebaree, H. Haron, and A. M. Abdulazeez, "Gene selection and classification of microarray data using convolutional neural network," in 2018 International Conference on Advanced Science and Engineering (ICOASE), 2018, pp. 145-150.
[43] A. S. Issa and A. M. A. Brifcani, "Intrusion detection and attack classifier based on three techniques: a comparative study," Engineering and Technology Journal, vol. 29, pp. 386-412, 2011.
[44] K. He and J. Sun, "Convolutional neural networks at constrained time cost," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5353-5360.
[45] P. Sharma, Y. P. S. Berwal, and W. Ghai, "Performance analysis of deep learning CNN models for disease detection in plants using image segmentation," Information Processing in Agriculture, vol. 7, pp. 566-574, 2020.
[46] F. Milletari, S.-A. Ahmadi, C. Kroll, A. Plate, V. Rozanski, J. Maiostre, et al., "Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound," Computer Vision and Image Understanding, vol. 164, pp. 92-102, 2017.
[47] M. A. Sulaiman, "Evaluating Data Mining Classification Methods Performance in Internet of Things Applications," Journal of Soft Computing and Data Mining, vol. 1, pp. 11-25, 2020.
[48] Y. Bengio, A. C. Courville, and P. Vincent, "Unsupervised feature learning and deep learning: A review and new perspectives," CoRR, abs/1206.5538, vol. 1, p. 2012, 2012.
[49] X. Zhao and M. Jia, "A novel unsupervised deep learning network for intelligent fault diagnosis of rotating machinery," Structural Health Monitoring, vol. 19, pp. 1745-1763, 2020.
[50] A. F. Jahwar and A. M. A. Duhok, "META-HEURISTIC ALGORITHMS FOR K-MEANS CLUSTERING: A REVIEW," PalArch's Journal of Archaeology of Egypt/Egyptology, vol. 17, pp. 12002-12020, 2020.
[51] G. E. Hinton, "A practical guide to training restricted Boltzmann machines," in Neural networks: Tricks of the trade, ed: Springer, 2012, pp. 599-619.
[52] H. Yu, Z. Tian, H. Li, B. Xu, and G. An, "A Novel Deep Belief Network Model Constructed by Improved Conditional RBMs and its Application in RUL Prediction for Hydraulic Pumps," International Journal of Acoustics and Vibration, vol. 25, pp. 373-382, 2020.
[53] H. Yao, L. Jiang, and E. Wei, "Enhanced Deep Learning Approach Based on the Deep Convolutional Encoder-Decoder Architecture for Electromagnetic Inverse Scattering Problems," IEEE Antennas and Wireless Propagation Letters, 2020.
[54] O. Karaoğlu, H. Ş. Bilge, and İ. Uluer, "Reducing Speckle Noise from Ultrasound Images Using an Autoencoder Network," in 2020 28th Signal Processing and Communications Applications Conference (SIU), 2020, pp. 1-4.
[55] E. A. Rashed, J. Gomez-Tames, and A. Hirata, "End-to-end semantic segmentation of personalized deep brain structures for non-invasive brain stimulation," Neural Networks, vol. 125, pp. 233-244, 2020.
[56] N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B. Gotway, et al., "Convolutional neural networks for medical image analysis: Full training or fine tuning?," IEEE transactions on medical imaging, vol. 35, pp. 1299-1312, 2016.
[57] J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," Journal of machine learning research, vol. 12, 2011.
[58] V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in ICML, 2010.
[59] X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier neural networks," in Proceedings of the fourteenth international conference on artificial intelligence and statistics, 2011, pp. 315-323.
[60] R. Pascanu, T. Mikolov, and Y. Bengio, "On the difficulty of training recurrent neural networks," in International conference on machine learning, 2013, pp. 1310-1318.
[61] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," arXiv preprint arXiv:1207.0580, 2012.
[62] S. Ioffe and C. Szegedy, "Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv e-prints," 2015.
[63] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, and L. Bottou, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," Journal of machine learning research, vol. 11, 2010.
[64] A. R. Jamieson, K. Drukker, and M. L. Giger, "Breast image feature learning with adaptive deconvolutional networks," in Medical Imaging 2012: Computer-Aided Diagnosis, 2012, p. 831506.
[65] H. Xie, N. Wang, M. He, L. Zhang, H. Cai, J. Xian, et al., "Using deep‐learning algorithms to classify fetal brain ultrasound images as normal or abnormal," Ultrasound in Obstetrics & Gynecology, vol. 56, pp. 579-587, 2020.
[66] L. Zhang, N. J. Dudley, T. Lambrou, N. Allinson, and X. Ye, "Automatic image quality assessment and measurement of fetal head in two-dimensional ultrasound image," Journal of Medical Imaging, vol. 4, p. 024001, 2017.
[67] L. Zhang, X. Ye, T. Lambrou, W. Duan, N. Allinson, and N. J. Dudley, "A supervised texton based approach for automatic segmentation and measurement of the fetal head and femur in 2D ultrasound images," Physics in Medicine & Biology, vol. 61, p. 1095, 2016.
[68] F. Isensee, P. F. Jaeger, S. A. Kohl, J. Petersen, and K. H. Maier-Hein, "nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation," Nature Methods, pp. 1-9, 2020.
[69] H. He and E. A. Garcia, "Learning from imbalanced data," IEEE Transactions on knowledge and data engineering, vol. 21, pp. 1263-1284, 2009.
[70] A.-I. Barranco-Gutiérrez, "Machine Learning for Brain Images Classification of Two Language Speakers," Computational Intelligence and Neuroscience, vol. 2020, 2020.
[71] P. Jagadeesh, S. Subashini, and R. V. Lakshmi, "Detecting and Classifying Fetal Brain Abnormalities Using Deep Learning."
[72] M. Toğaçar, B. Ergen, and Z. Cömert, "BrainMRNet: Brain tumor detection using magnetic resonance images with a novel convolutional neural network model," Medical Hypotheses, vol. 134, p. 109531, 2020.
[73] R. Qu, G. Xu, C. Ding, W. Jia, and M. Sun, "Deep Learning-Based Methodology for Recognition of Fetal Brain Standard Scan Planes in 2D Ultrasound Images," IEEE Access, vol. 8, pp. 44443-44451, 2019.
[74] J. M. J. Valanarasu, R. Yasarla, P. Wang, I. Hacihaliloglu, and V. M. Patel, "Learning to segment brain anatomy from 2D ultrasound with less data," IEEE Journal of Selected Topics in Signal Processing, vol. 14, pp. 1221-1234, 2020.
[75] E. L. Skeika, M. R. Da Luz, B. J. T. Fernandes, H. V. Siqueira, and M. L. S. C. De Andrade, "Convolutional Neural Network to Detect and Measure Fetal Skull Circumference in Ultrasound Imaging," IEEE Access, vol. 8, pp. 191519-191529, 2020.
[76] A. I. Namburete, R. V. Stebbing, B. Kemp, M. Yaqub, A. T. Papageorghiou, and J. A. Noble, "Learning-based prediction of gestational age from ultrasound images of the fetal brain," Medical image analysis, vol. 21, pp. 72-86, 2015.
[77] M. Hamisa, N. Dabees, W. M. Ataalla, and D. H. Ziada, "Magnetic resonance imaging versus Ultrasound examination in detection of prenatal fetal brain anomalies," The Egyptian Journal of Radiology and Nuclear Medicine, vol. 44, pp. 665-672, 2013.
[78] Z. Lin, S. Li, D. Ni, Y. Liao, H. Wen, J. Du, et al., "Multi-task learning for quality assessment of fetal head ultrasound images," Medical image analysis, vol. 58, p. 101548, 2019.
[79] S. S. M. Salehi, S. R. Hashemi, C. Velasco-Annis, A. Ouaalam, J. A. Estroff, D. Erdogmus, et al., "Real-time automatic fetal brain extraction in fetal MRI by deep learning," in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 2018, pp. 720-724.
[80] C. F. Baumgartner, K. Kamnitsas, J. Matthew, T. P. Fletcher, S. Smith, L. M. Koch, et al., "SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound," IEEE transactions on medical imaging, vol. 36, pp. 2204-2215, 2017.
[81] P. Sridar, A. Kumar, A. Quinton, D. Feng, K. Ramarathnam, R. Nanan, et al., "Automatic identification of multiple planes of a fetal organ from 2-d ultrasound images," in Proc. MICCAI Workshop Perinatal, Preterm Paediatric Image Anal.(PIPPI), 2016, pp. 1-10.
[82] G. Sanroma, O. M. Benkarim, G. Piella, K. Lekadir, N. Hahner, E. Eixarch, et al., "Learning to combine complementary segmentation methods for fetal and 6-month infant brain MRI segmentation," Computerized Medical Imaging and Graphics, vol. 69, pp. 52-59, 2018.
[83] L. Xu, M. Liu, J. Zhang, and Y. He, "Convolutional-Neural-Network-Based Approach for Segmentation of Apical Four-Chamber View from Fetal Echocardiography," IEEE Access, vol. 8, pp. 80437-80446, 2020.
[84] B. Al-Bander, T. Alzahrani, S. Alzahrani, B. M. Williams, and Y. Zheng, "Improving Fetal Head Contour Detection by Object Localisation with Deep Learning," in Annual Conference on Medical Image Understanding and Analysis, 2019, pp. 142-150.
[85] J. Li, Y. Luo, L. Shi, X. Zhang, M. Li, B. Zhang, et al., "Automatic fetal brain extraction from 2D in utero fetal MRI slices using deep neural network," Neurocomputing, vol. 378, pp. 335-349, 2020.
[86] W. Loh, A. Adno, and S. Reid, "VP59. 30: Spontaneous tubal stump ectopic pregnancy following ipsilateral salpingectomy," Ultrasound in Obstetrics & Gynecology, vol. 56, pp. 322-322, 2020.
[87] S. S. M. Salehi, D. Erdogmus, and A. Gholipour, "Auto-context convolutional neural network (auto-net) for brain extraction in magnetic resonance imaging," IEEE transactions on medical imaging, vol. 36, pp. 2319-2330, 2017.
[88] R. Huang, W. Xie, and J. A. Noble, "VP-Nets: Efficient automatic localization of key brain structures in 3D fetal neurosonography," Medical image analysis, vol. 47, pp. 127-139, 2018.
[89] B. M. Williams and Y. Zheng, "Improving Fetal Head Contour Detection by Object Localisation with Deep Learning," in Medical Image Understanding and Analysis: 23rd Conference, MIUA 2019, Liverpool, UK, July 24-26, 2019, Proceedings, 2020, p. 142.
[90] P. Deepika, R. Suresh, and P. Pabitha, "Defending Against Child Death: Deep learning‐based diagnosis method for abnormal identification of fetus ultrasound Images," Computational Intelligence, 2020.
[91] H. Xie, N. Wang, M. He, L. Zhang, H. Cai, J. Xian, et al., "Using deep learning algorithms to classify fetal brain ultrasound images as normal or abnormal," Ultrasound in Obstetrics & Gynecology, 2020.
