Skip to main content

Advertisement

SpringerLink
Log in

Breast lesion classification from mammograms using deep neural network and test-time augmentation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In order to effectively aid in the automatic classification of the breast cancer suspicious region, a new deep-learning (DL) model built on the transfer-learning (TL) technique is proposed in this work. Pretrained convolutional neural networks (CNN) architectures, such as VGG-16, ResNet-50, Inception-V3, and Efficientnet-B7 are used in the proposed model to extract the features from the MIAS and CBIS-DDSM datasets. Three scenarios are used to evaluate the performance of each network under consideration: TL for the original dataset, TL for the pre-processed dataset, and a novel approach of TL with test time augmentation (TTA). Concerning limited training datasets, the proposed approach showed that pre-trained classification networks with test time augmentation are much more effective and efficient, making them more acceptable for medical imaging. With accuracy (99.97%), specificity (99.24%), sensitivity (98.50%), and F1-score (98.74%), the proposed test time augmentation strategy with transfer learning outperforms other state-of-the-art methods on the MIAS dataset. The proposed model could also do well on the CBIS-DDSM dataset with accuracy (99.8%), specificity (95.44%), sensitivity (96.57%), and F1-score (97.22%). Additionally, on the mammography predictions provided by our algorithm, we also sought the advice of a radiology professional. The model’s predictions and the expert’s assessment are almost in sync.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Abbreviations

DL:

Deep learning

TL:

Transfer learning

TTA:

Test time augmentation

CADD:

Computer-assisted detection and diagnosis

CAD:

Computer aided diagnosis

DCNN:

Deep convolutional neural network

CNN:

Convolutional neural network

FC:

Fully connected

MBConv:

Mobile inverted bottleneck convolution

CLAHE:

Contrast limited adaptive histogram equalization

FLOP:

Floating point operation

RoI:

Region of interest

ANOVA:

Analysis of variance

HSD:

Honestly significantly differenced

References

  1. Cancer Facts and Figures 2021. https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2021.html. Accessed 31 Mar 2022 (2021)

  2. Mehrotra R, Yadav K (2022) Breast cancer in india: present scenario and the challenges ahead. World J Clin Oncol 13(3):209

    Article  Google Scholar 

  3. Alarming facts about breast cancer in India. https://www.oncostem.com/blog/alarming-facts-about-breast-cancer-in-india. Accessed 26 Mar 2022 (2019)

  4. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A (2018) Global cancer statistics 2018: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 68(6):394–424

    Article  Google Scholar 

  5. Oza P, Sharma P, Patel S (2021) Machine learning applications for computer-aided medical diagnostics. In: Proceedings of second international conference on computing, communications, and cyber-security, pp 377–392 (2021). Springer

  6. Oza P, Sharma P, Patel S (2022) A drive through computer-aided diagnosis of breast cancer: a comprehensive study of clinical and technical aspects. Recent Innov Comput 1:233–249

    Article  Google Scholar 

  7. Pillai R, Oza P, Sharma P (2020) Review of machine learning techniques in health care. In: Proceedings of ICRIC 2019, pp 103–111. Springer

  8. Tsochatzidis L, Costaridou L, Pratikakis I (2019) Deep learning for breast cancer diagnosis from mammograms-a comparative study. J Imaging 5(3):37

    Article  Google Scholar 

  9. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25 (2012)

  10. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 (2016)

  12. Tan M, Le Q (2019) Efficientnet: rethinking model scaling for convolutional neural networks. In: International conference on machine learning, pp 6105–6114. PMLR

  13. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9 (2015)

  14. Oza P, Sharma P, Patel S, Bruno A (2021) A bottom-up review of image analysis methods for suspicious region detection in mammograms. J Imaging. https://doi.org/10.3390/jimaging7090190

    Article  Google Scholar 

  15. Wang G, Li W, Aertsen M, Deprest J, Ourselin S, Vercauteren T (2019) Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing 338:34–45

    Article  Google Scholar 

  16. Oza P, Sharma P, Patel S, Kumar P (2022) Deep convolutional neural networks for computer-aided breast cancer diagnostic: a survey. Neural Comput Appl 34:1815–1836. https://doi.org/10.1007/s00521-021-06804-y

    Article  Google Scholar 

  17. Guan S, Loew M (2017) Breast cancer detection using transfer learning in convolutional neural networks. In: 2017 IEEE applied imagery pattern recognition workshop (AIPR), pp 1–8. IEEE

  18. Altaf MM (2021) A hybrid deep learning model for breast cancer diagnosis based on transfer learning and pulse-coupled neural networks. Math Biosci Eng 18(5):5029–5046

    Article  MathSciNet  Google Scholar 

  19. Alruwaili M, Gouda W (2022) Automated breast cancer detection models based on transfer learning. Sensors 22(3):876

    Article  Google Scholar 

  20. Sha Z, Hu L, Rouyendegh BD (2020) Deep learning and optimization algorithms for automatic breast cancer detection. Int J Imaging Syst Technol 30(2):495–506

    Article  Google Scholar 

  21. Saber A, Sakr M, Abo-Seida OM, Keshk A, Chen H (2021) A novel deep-learning model for automatic detection and classification of breast cancer using the transfer-learning technique. IEEE Access 9:71194–71209

    Article  Google Scholar 

  22. Carneiro G, Nascimento J, Bradley AP (2015) Unregistered multiview mammogram analysis with pre-trained deep learning models. In: International conference on medical image computing and computer-assisted intervention, pp 652–660. Springer

  23. Shams S, Platania R, Zhang J, Kim J, Lee K, Park SJ (2018) Deep generative breast cancer screening and diagnosis. In: International conference on medical image computing and computer-assisted intervention, pp 859–867. Springer

  24. Jabeen KEA (2023) Bc2netrf: breast cancer classification from mammogram images using enhanced deep learning features and equilibrium-jaya controlled regula falsi-based features selection. Diagnostics 13(7):1238

    Article  Google Scholar 

  25. Atban F, Ekinci E, Garip Z (2023) Traditional machine learning algorithms for breast cancer image classification with optimized deep features. Biomed Signal Process Control 81:104534

    Article  Google Scholar 

  26. Pereira JMS, de Santana MA, de Lima CL, de Lima RdCF., de Lima SML, dos Santos WP (2023) Feature selection based on dialectical optimization algorithm for breast lesion classification in thermographic images. In: Research anthology on medical informatics in breast and cervical cancer, pp 96–119. IGI Global (2023)

  27. Oza P, Sharma P, Patel S (2023) Deep ensemble transfer learning-based framework for mammographic image classification. J Supercomput 79(7):8048–8069

    Article  Google Scholar 

  28. Hamidinekoo A, Denton E, Rampun A, Honnor K, Zwiggelaar R (2018) Deep learning in mammography and breast histology, an overview and future trends. Med Image Anal 47:45–67

    Article  Google Scholar 

  29. Oza P, Shah Y, Vegda M (2022) A comprehensive study of mammogram classification techniques. In: Tracking and preventing diseases with artificial intelligence, pp 217–238. Springer (2022)

  30. Arevalo J, González FA, Ramos-Pollán R, Oliveira JL, Lopez MAG (2016) Representation learning for mammography mass lesion classification with convolutional neural networks. Comput Methods Programs Biomed 127:248–257

    Article  Google Scholar 

  31. Huynh BQ, Li H, Giger ML (2016) Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. J Med Imaging 3(3):034501

    Article  Google Scholar 

  32. Lévy D, Jain A (2016) Breast mass classification from mammograms using deep convolutional neural networks. arXiv preprint arXiv:1612.00542

  33. Oza P, Sharma P, Patel S (2022) Transfer learning assisted classification of artefacts removed and contrast improved digital mammograms. Scalable Comput Pract Exp 23(3):115–127

    Article  Google Scholar 

  34. Oza P, Sharma P, Patel S (2022) A transfer representation learning approach for breast cancer diagnosis from mammograms using efficientnet models. Scalable Comput Pract Exp 23(2):51–58

    Article  Google Scholar 

  35. Khan S, Islam N, Jan Z, Din IU, Rodrigues JJC (2019) A novel deep learning based framework for the detection and classification of breast cancer using transfer learning. Pattern Recog Lett 125:1–6

    Article  Google Scholar 

  36. Agarwal R, Díaz O, Yap MH, Llado X, Marti R (2020) Deep learning for mass detection in full field digital mammograms. Comput Biol Med 121:103774

    Article  Google Scholar 

  37. Vedalankar AV, Gupta SS, Manthalkar RR (2021) Addressing architectural distortion in mammogram using alexnet and support vector machine. Inform Med Unlocked 23:100551

    Article  Google Scholar 

  38. Kim I, Kim Y, Kim S (2020) Learning loss for test-time augmentation. Adv Neural Inf Process Syst 33:4163–4174

    Google Scholar 

  39. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241. Springer

  40. Isensee F, Petersen J, Klein A, Zimmerer D, Jaeger PF, Kohl S, Wasserthal J, Koehler G, Norajitra T, Wirkert S et al (2018) nnu-net: self-adapting framework for u-net-based medical image segmentation. arXiv preprint arXiv:1809.10486

  41. Pang T, Xu K, Zhu J (2019) Mixup inference: better exploiting mixup to defend adversarial attacks. arXiv preprint arXiv:1909.11515

  42. Suckling J, Parker J, Dance D, Astley S, Hutt I, Boggis C, Ricketts I, Stamatakis E, Cerneaz N, Kok S, et al (2015) Mammographic image analysis society (mias) database v1. 21

  43. Lee RS, Gimenez F, Hoogi A, Miyake KK, Gorovoy M, Rubin DL (2017) A curated mammography data set for use in computer-aided detection and diagnosis research. Sci Data 4(1):1–9

    Article  Google Scholar 

  44. Heath M, Bowyer K, Kopans D, Kegelmeyer P, Moore R, Chang K, Munishkumaran S (1998) Current status of the digital database for screening mammography. In: Digital mammography, pp 457–460. Springer

  45. He K, Sun J (2015) Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5353–5360 (2015)

  46. Srivastava RK, Greff K, Schmidhuber J (2015) Highway networks. arXiv preprint arXiv:1505.00387

  47. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520 (2018)

  48. Oza P, Sharma P, Patel S, Adedoyin F, Bruno A (2022) Image augmentation techniques for mammogram analysis. J Imaging. https://doi.org/10.3390/jimaging8050141

    Article  Google Scholar 

  49. Gu K, Zhai G, Lin W, Liu M (2015) The analysis of image contrast: from quality assessment to automatic enhancement. IEEE Trans Cybern 46(1):284–297

    Article  Google Scholar 

  50. Xing H, Xiao Z, Qu R, Zhu Z, Zhao B (2022) An efficient federated distillation learning system for multitask time series classification. IEEE Trans Instrum Meas 71:1–12

    Google Scholar 

  51. Xing H, Xiao Z, Zhan D, Luo S, Dai P, Li K (2022) Selfmatch: robust semisupervised time-series classification with self-distillation. Int J Intell Syst 37(11):8583–8610

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Dr. Rajiv Oza (Consultant Radiologist) for helping them validate the model prediction findings. The authors also thank Department of Computer Science and Engineering, Nirma University, Ahmedabad, for providing computing facilities to carry out this work.

Author information

Authors and Affiliations

  1. Nirma University, Ahmedabad, Gujarat, 382481, India

    Parita Oza

  2. Pandit Deendayal Energy University, Gandhinagar, Gujarat, 382007, India

    Paawan Sharma & Samir Patel

Authors
  1. Parita Oza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paawan Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Samir Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Parita Oza.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oza, P., Sharma, P. & Patel, S. Breast lesion classification from mammograms using deep neural network and test-time augmentation. Neural Comput & Applic 36, 2101–2117 (2024). https://doi.org/10.1007/s00521-023-09165-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09165-w

Keywords

Search

Navigation