This paper provides a systematic review of augmented reality (AR) technologies and their applications in medical diagnosis and treatment from 1988-2023. The paper is categorised by key technologies of augmented reality and medical diagnosis and treatment applications. For AR key technologies, 67 articles were researched. Among them, AR key technologies can be divided into three methods: display, calibration and tracking, with video perspective display technology dominating the field. For calibration technologies, 17 articles were included in the review, and we discussed methods based on graphic markers, optical markers and markerless calibration, respectively; for positional tracking technologies, 15 articles were counted, and we discussed methods based on markers, simultaneous localisation and map building, and optical flow-based tracking, respectively; for application areas of AR technologies, simulation teaching and training, surgical guidance and telemedicine were included in the review Scope.

Du Wenjia, Yan Ziqi. Application and development of augmented reality in the medical field. International Journal of Frontiers in Medicine (2023), Vol. 5, Issue 6: 65-76. https://doi.org/10.25236/IJFM.2023.050611.

[1] Blanco-Pons, S., B. Carrión-Ruiz and J.L. Lerma, Augmented reality application assessment for disseminating rock art. Multimedia Tools and Applications, 2019. 78(8): p. 10265-10286.

[2] Zhou Z, Zhou Yi & Xiao Jiangjian, A review of virtual reality augmentation technologies. Chinese Science:Information Science, 2015. 45(02): pp. 157-180.

[3] Zhang Sifang & Jiang Jiafa, Research and prospect of teaching application of augmented reality technology under the perspective of science education. Electrochemical Education Research, 2018. 39(07): pp. 64-69+90.

[4] Jung-Hwan, K., et al., Immersive interactive technologies and virtual shopping experiences: Differences in consumer perceptions between augmented Telematics and Informatics, 20(1): 3. Telematics and Informatics, 2023. 77.

[5] Yin S. Jia, Exploring the technological advancement and development prospects of AR and VR. Contemporary Educational Practice and Teaching Research, 2020(04): pp. 243-244.

[6] Wu, Binhai et al. A review of projection display technology for augmented reality surgical navigation systems. Optical Precision Engineering, 2021. 29(09): pp. 2019-2038.

[7] Wang, J., et al., Real-Time Marker-Free Patient Registration and Image-Based Navigation Using Stereovision for Dental Surgery, H. Liao, et al. , et al.^Editors. 2013, Springer Berlin Heidelberg: Berlin, Heidelberg. p. 9-18.

[8] Ai, D., et al., Augmented reality based real-time subcutaneous vein imaging system. Biomed Opt Express, 2016. 7(7): p. 2565-85.

[9] Wu, J., et al., Real-time advanced spinal surgery via visible patient model and augmented reality system. computer Methods and Programs in Biomedicine. 2014. 113(3): p. 869-881.

[10] Peng, L., et al., A co-axial projection surgical navigation system for breast cancer sentinel lymph node mapping: system design and clinical trial. biOS , 2019.

[11] Wu Shouxuan et al. In situ coaxial projection for neurosurgical navigation. China Laser, 2020. 47(02): pp. 387-394.

[12] T., S., F. M. and N. N., Advanced Medical Displays: A Literature Review of Augmented Reality. Journal of Display Technology, 2008. 4(4): p. 451-467.

[13] Liu, J., et al., An augmented reality system for image guidance of transcatheter procedures for structural heart disease. PLOS ONE, 2019. 14(7): p. e0219174.

[14] S., L.P., et al. A Mixed-Reality System for Breast Surgical Planning. in 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR- Adjunct). 2017.

[15] Frantz, T., et al., Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation. Healthc Technol Lett, 2018. 5(5): p. 221-225.

[16] Murugesan, Y.P., et al., A novel rotational matrix and translation vector algorithm: geometric accuracy for augmented reality in oral and maxillofacial surgeries. The International Journal of Medical Robotics and Computer Assisted Surgery, 2018. 14(3): p. e1889.

[17] Xie G et al. Value of simple smartphone-based augmented reality for localization of supratentorial hypertensive intracerebral hematoma. Chinese Journal of Clinical Neurosurgery, 2019. 24(06): pp. 345-347+351.

[18] Tran, H.H., et al., Augmented reality system for oral surgery using 3D auto stereoscopic visualization. Med Image Comput Comput Assist Interv, 2011. 14( Pt 1): p. 81-8. Med Image Comput Assist Interv, 2011. 14(Pt 1): p. 81-8.

[19] Schneider, C., et al., Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility Surgical Endoscopy, 2020. 34(10): p. 4702-4711.

[20] Hussain, R., et al., Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery. Reports, 2020. 10(1): p. 6767.

[21] Pflugi, S., et al., Augmented marker tracking for peri-acetabular osteotomy surgery. International Journal of Computer Assisted Radiology and Surgery, 2018. 13(2): p. 291-304.

[22] Zhu, M., et al., Registration Strategy Using Occlusal Splint Based on Augmented Reality for Mandibular Angle Oblique Split Osteotomy. Journal of Craniofacial Surgery, 2011. 22(5): p. 1806-1809.

[23] Lin, Y., et al., A Novel Dental Implant Guided Surgery Based on Integration of Surgical Template and Augmented Reality. Clinical Implant Dentistry and Related Research, 2015. 17(3): p. 543-553.

[24] Ahn, J., et al., Tracking Accuracy of a Stereo Camera-Based Augmented Reality Navigation System for Orthognathic Surgery. Journal of Oral and Maxillofacial Surgery, 2019. 77(5).

[25] Ma, L., et al., Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study. International Journal of Computer Assisted Radiology and Surgery, 2017. 12(12): p. 2205-2215.

[26] Tran, H.H., et al., Augmented reality system for oral surgery using 3D auto stereoscopic visualization. med Image Comput Comput Assist Interv, 2011. 14( Pt 1): p. 81-8. Med Image Comput Assist Interv, 2011. 14(Pt 1): p. 81-8.

[27] Y, C., et al., Cone-beam computed tomography on a mobile C-arm: novel intraoperative imaging technology for guidance of head and neck surgery. Journal of otolaryngology, 2008. 37(1).

[28] A., K.E., et al., CT Fluoroscopy for Image-Guided Procedures: Physician Radiation Dose During Full-Rotation and Partial-Angle CT Scanning. Journal of Vascular and Interventional Radiology, 2021. 32(3).

[29] Brouwer, O.R., et al. Image navigation as a means to expand the boundaries of fluorescence-guided surgery. Physics in Medicine and Biology, 2012. 57(10).

[30] Primo, M., et al., Robust gradient-based 3-D/2-D registration of CT and MR to X-ray images. IEEE transactions on medical imaging, 2008. 27(12).

[31] A, M.R., et al., A comparison of a similarity-based and a feature-based 2-D-3-D registration method for neurointerventional use. IEEE transactions on medical imaging, 2005. 24(8).

[32] Livyatan, H., Z. Yaniv and L. Joskowicz, Gradient-based 2-D/3-D rigid registration of fluoroscopic X-ray to CT. IEEE Transactions on Medical Imaging. 2003. 22(11): p. 1395-1406.

[33] Tomazevic, D., et al., 3-D/2-D registration of CT and MR to X-ray images. IEEE Transactions on Medical Imaging, 2003. 22(11): p. 1407-1416.

[34] Hipwell, J.H., et al., Intensity-based 2-D-3-D registration of cerebral angiograms. IEEE Transactions on Medical Imaging, 2003. 22(11): p. 1417-1426 .

[35] McLaughlin, R.A., et al. A comparison of a similarity-based and a feature-based 2-D-3-D registration method for neurointerventional use. IEEE Transactions on Medical Imaging, 2005. 24(8): p. 1058-1066.

[36] Chung, A.C.S., et al. Multi-modal Image Registration by Minimising Kullback-Leibler Distance. 2002. Berlin, Heidelberg: Springer Berlin Heidelberg .

[37] Tristan P. C. Van Doormaal, et al., Clinical Accuracy of Holographic Navigation Using Point-Based Registration on Augmented-Reality Glasses. INSTRUMENTATION AND TECHNIQUE, 2018.

[38] Nicolau, S.A., et al., An augmented reality system for liver thermal ablation: Design and evaluation on clinical cases. Medical Image Analysis, 2009. 13 (3): p. 494-506.

[39] Fischer, J., et al., A hybrid tracking method for surgical augmented reality. Computers & Graphics, 2007. 31(1): p. 39-52.

[40] Chen Qingmei, Design and implementation of SLAM parameter adaptive adjustment system based on deep reinforcement learning, 2022, Guizhou University.

[41] Ma, C., et al., Moving-Tolerant Augmented Reality Surgical Navigation System Using Autostereoscopic Three-Dimensional Image Overlay. IEEE Journal of Biomedical and Health Informatics, 2019. 23(6): p. 2483-2493.

[42] M, S., et al., Development of an endoscopic navigation system based on digital image processing. computer aided surgery : official journal of the International Society for Computer Aided Surgery, 1998. 3(3).

[43] Y, K., et al., An articulated neurosurgical navigation system using MRI and CT images. IEEE transactions on bio-medical engineering, 1988. 35(2).

[44] Wang, H., et al., Robust Motion Estimation and Structure Recovery from Endoscopic Image Sequences With an Adaptive Scale Kernel Consensus Estimator. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 2008. 2008: p. 1-7.

[45] Luó, X., et al., Development and comparison of new hybrid motion tracking for bronchoscopic navigation. Medical Image Analysis, 2012. 16(3): p. 577-596 .

[46] Nagao, J., et al. Fast and Accurate Bronchoscope Tracking Using Image Registration and Motion Prediction. 2004. Berlin, Heidelberg: Springer Berlin Heidelberg.

[47] Qin Yue, Research on motion target detection and tracking algorithms based on optical flow features, 2013, Beijing Institute of Technology.

[48] Liu, J., K.R. Subramanian and T.S. Yoo, An optical flow approach to tracking colonoscopy video. Computerized Medical Imaging and Graphics, 2013. 37(3). P. 207-223.

[49] Xu, Q., et al., Lung tumor tracking in fluoroscopic video based on optical flow. Med Phys, 2008. 35(12): p. 5351-9.

[50] H., Z., et al. Polyp Tracking in Video Colonoscopy Using Optical Flow With an On-The-Fly Trained CNN. in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). 2019.

[51] Ling Guanhua et al, A preliminary investigation on the construction of a virtual simulation experimental teaching centre based on VR and AR technologies at the Army Medical University. Value Engineering, 2018. 37(32): pp. 252-254.

[52] Liu Zhuohui & Long Ruiqing, Characteristics of the times and main features of medical education in the new era and development trends. Curriculum Education Research, 2017(45): p. 241.

[53] Yuan B et al. Evaluation of the application and effectiveness of AR-based navigation technology for pedicle screw placement in teaching spine surgeon trainees. Journal of Naval Medicine, 2022. 43(10): pp. 1157-1159.

[54] Wu WJ et al. Application of augmented reality technology combined with flipped classroom style teaching method in teaching clinical cardiopulmonary resuscitation to nurses. Guangxi Medicine, 2022. 44(19): pp. 2325-2328.

[55] Zhang Xuanping, Application and exploration of virtual reality acupuncture teaching system. Hebei Journal of Traditional Chinese Medicine, 2022. 37(01): pp. 61-64.

[56] Wang QF et al. Augmented reality-based navigation system for minimally invasive spine surgery. Robotics: pp. 1-8.

[57] Zhou Y et al. Early study of "augmented reality"-assisted precision thoracoscopic resection of small pulmonary nodules. Chinese Journal of Medical Devices, 2022. 46(06): pp. 607-610.

[58] Suenaga, H., et al., Real-time in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study. International Journal of Oral Science, 2013. 5(02): p. 98-102.

[59] Wang, Tianmiao et al. A review of research advances in laparoscopic augmented reality navigation. Robotics, 2019. 41(01): pp. 124-136.

[60] Robotics; Findings from IRCAD Provide New Insights into Robotics (Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy). Journal of Engineering, 2015.

[61] Shinji, O., et al., Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality-based navigation system. journal of hepato-biliary-pancreatic sciences, 2014. 21(4).

[62] Tomoyoshi, O., et al., Utility of augmented reality system in hepatobiliary surgery. journal of hepato-biliary-pancreatic sciences, 2013. 20(2).

[63] Shi Yangyang, Holographic augmented reality-based precision puncture guidance for heterogeneous tumors in the liver, 2022, Wuhan Institute of Post and Telecommunications Science.

[64] Peng, L., et al., A Wearable Augmented Reality Navigation System for Surgical Telementoring Based on Microsoft HoloLens. Annals of biomedical engineering, 2020. 49(prepublished).

[65] Xiao Ruoxiu et al. Clinical application of AR tele-guidance system in the fight against novel coronavirus pneumonia outbreak. Medical and Health Equipment, 2020. 41(07): pp. 7-12.

[66] Dan, A., et al., An Augmented Reality-Based Approach for Surgical Telementoring in Austere Environments. military medicine, 2017. 182(S1).

[67] Andersen, D., et al., Medical telementoring using an augmented reality transparent display. surgery, 2016. 159(6).