Emerging Challenges in Technology-based Support for Surgical Training
Abstract
This paper stipulates several technological research and development thrusts that can assist in modern day approaches to simulated training of minimally invasive laparoscopic and robot surgery. Basic tenets of such training are explained, and specific areas of research are enumerated. Specifically, augmented and mixed reality are proposed as a means of improving perceptual and clinical decision-making skills, haptics are proposed as mechanism not only to provide force feedback and guidance, but also as a means of reflecting a tactile feel of surgery in simulated training scenarios. Learning optimization is discussed to fine tune the difficulty levels of various exercises. All the above elements can serve as the foundation for building computer-based virtual coaching environments that can reduce the training costs and provide a broader access to learning highly complex, technology driven surgical techniques.
References
W. Lau, C. Leow, and A. Li, “History of endoscopic and laparoscopic surgery,” World journal of surgery, vol. 21, no. 4, p. 444, 1997.
I. Nisky, F. Huang, A. Milstein, C. M. Pugh, F. A. Mussa-Ivaldi, and A. Karniel, “Perception of stiffness in laparoscopy–the fulcrum effect,” Studies in health technology and informatics, vol. 173, p. 313, 2012.
T. Leal Ghezzi and O. Campos Corleta, “30 years of robotic surgery,” World journal of surgery, vol. 40, pp. 2550–2557, 2016.
C. Freschi, V. Ferrari, F. Melfi, M. Ferrari, F. Mosca, and A. Cuschieri, “Technical review of the da vinci surgical telemanipulator,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 9, no. 4, pp. 396–406, 2013.
N. J. Soper and G. M. Fried, “The fundamentals of laparoscopic surgery: its time has come,” Bull Am Coll Surg, vol. 93, no. 9, pp. 30–32, 2008.
N. Stylopoulos, S. Cotin, S. Maithel, M. Ottensmeyer, P. Jackson, R. Bardsley, P. Neumann, D. Rattner, and S. Dawson, “Computer-enhanced laparoscopic training system (celts): bridging the gap,” Surgical Endoscopy and Other Interventional Techniques, vol. 18, pp. 782–789, 2004.
A. S. Soyinka, T. Schollmeyer, I. Meinhold-Heerlein, D. V. Gopalghare, H. Hasson, and L. Mettler, “Enhancing laparoscopic performance with the lts3e: a computerized hybrid physical reality simulator,” Fertility and sterility, vol. 90, no. 5, pp. 1988–1994, 2008.
S. Cotin, N. Stylopoulos, M. Ottensmeyer, P. Neumann, D. Rattner, and S. Dawson, “Metrics for laparoscopic skills trainers: The weakest link!” in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2002: 5th International Conference Tokyo, Japan, September 25–28, 2002 Proceedings, Part I 5. Springer, 2002, pp. 35–43.
P. B. Andreatta, D. T. Woodrum, J. D. Birkmeyer, R. K. Yellamanchilli, G. M. Doherty, P. G. Gauger, and R. M. Minter, “Laparoscopic skills are improved with lapmentor™ training: results of a randomized, double-blinded study,” Annals of surgery, vol. 243, no. 6, p. 854, 2006.
K. Kawaguchi, H. Egi, M. Hattori, H. Sawada, T. Suzuki, and H. Ohdan, “Validation of a novel basic virtual reality simulator, the lap-x, for training basic laparoscopic skills,” Minimally Invasive Therapy & Allied Technologies, vol. 23, no. 5, pp. 287–293, 2014.
C. Basdogan, S. De, J. Kim, M. Muniyandi, H. Kim, and M. A. Srinivasan, “Haptics in minimally invasive surgical simulation and training,” IEEE computer graphics and applications, vol. 24, no. 2, pp. 56–64, 2004.
R.-J. Chen, H.-W. Lin, Y.-H. Chang, C.-T. Wu, and S.-T. Lee, “Development of an augmented reality force feedback virtual surgery training platform,” International Journal of Automation and Smart Technology, vol. 1, no. 1, pp. 41–51, 2011.
R. Smith, V. Patel, and R. Satava, “Fundamentals of robotic surgery: a course of basic robotic surgery skills based upon a 14-society consensus template of outcomes measures and curriculum development,” The international journal of medical robotics and computer assisted surgery, vol. 10, no. 3, pp. 379–384, 2014.
A. J. Hung, I. S. Jayaratna, K. Teruya, M. M. Desai, I. S. Gill, and A. C. Goh, “Comparative assessment of three standardized robotic surgery training methods,” BJU international, vol. 112, no. 6, pp. 864–871, 2013.
J. D. Bric, D. C. Lumbard, M. J. Frelich, and J. C. Gould, “Current state of virtual reality simulation in robotic surgery training: a review,” Surgical endoscopy, vol. 30, pp. 2169–2178, 2016.
S. Puliatti, E. Mazzone, and P. Dell’Oglio, “Training in robot-assisted surgery,” Current Opinion in Urology, vol. 30, no. 1, pp. 65–72, 2020.
M. Hong, J. W. Rozenblit, and A. J. Hamilton, “Simulation-based surgical training systems in laparoscopic surgery: a current review,” Virtual Reality, vol. 25, pp. 491–510, 2021.
L. St-Martin, P. Patel, J. Gallinger, and C.-a. Moulton, “Teaching the slowing-down moments of operative judgment,” Surgical Clinics, vol. 92, no. 1, pp. 125–135, 2012.
C.-a. Moulton, G. Regehr, L. Lingard, C. Merritt, and H. MacRae, “‘slowing down when you should’: initiators and influences of the transition from the routine to the effortful,” Journal of Gastrointestinal Surgery, vol. 14, pp. 1019–1026, 2010.
M. Speicher, B. D. Hall, and M. Nebeling, “What is mixed reality?” in Proceedings of the 2019 CHI conference on human factors in computing systems, 2019, pp. 1–15.
Q. Feng, H. P. Shum, and S. Morishima, “Resolving hand-object occlusion for mixed reality with joint deep learning and model optimization,” Computer Animation and Virtual Worlds, vol. 31, no. 4-5, p. e1956,2020.
S. Rokhsaritalemi, A. Sadeghi-Niaraki, and S.-M. Choi, “A review on mixed reality: Current trends, challenges and prospects,” Applied Sciences, vol. 10, no. 2, p. 636, 2020.
X. Zhao, G. Wang, Z. He, and H. Jiang, “A survey of moving object detection methods: A practical perspective,” Neurocomputing, vol. 503, pp. 28–48, 2022.
A. M. Schreiber, M. Hong, and J. W. Rozenblit, “Monocular depth estimation using synthetic data for an augmented reality training system in laparoscopic surgery,” in 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2021, pp. 2121–2126.
A. Hernansanz, D. Zerbato, L. Gasperotti, M. Scandola, P. Fiorini, and A. Casals, “Improving the development of surgical skills with virtual fixtures in simulation,” in Information Processing in Computer-Assisted Interventions: Third International Conference, IPCAI 2012, Pisa, Italy, June 27, 2012. Proceedings 3. Springer, 2012, pp. 157–166.
K. Tagawa, H. T. Tanaka, Y. Kurumi, M. Komori, and S. Morikawa, “Laparoscopic surgery simulator using first person view and guidance force,” in Medicine Meets Virtual Reality 20. IOS Press, 2013, pp. 431–435.
A. Bettini, P. Marayong, S. Lang, A. M. Okamura, and G. D. Hager, “Vision-assisted control for manipulation using virtual fixtures,” IEEE Transactions on Robotics, vol. 20, no. 6, pp. 953–966, 2004.
J. van Oosterhout, J. G. Wildenbeest, H. Boessenkool, C. J. Heemskerk, M. R. de Baar, F. C. van der Helm, and D. A. Abbink, “Haptic shared control in tele-manipulation: Effects of inaccuracies in guidance on task execution,” IEEE transactions on haptics, vol. 8, no. 2, pp. 164–175, 2015.
M. Hong and J. W. Rozenblit, “An adaptive force guidance system for computer-guided laparoscopy training,” IEEE Transactions on Cybernetics, vol. 52, no. 8, pp. 8019–8031, 2021.
S. A. Bowyer, B. L. Davies, and F. R. y Baena, “Active constraints/virtual fixtures: A survey,” IEEE Transactions on Robotics, vol. 30, no. 1, pp. 138–157, 2013.
I. A. Sucan, M. Moll, and L. E. Kavraki, “The open motion planning library,” IEEE Robotics & Automation Magazine, vol. 19, no. 4, pp. 72–82, 2012.
A. Paraschos, E. Rueckert, J. Peters, and G. Neumann, “Probabilistic movement primitives under unknown system dynamics,” Advanced Robotics, vol. 32, no. 6, pp. 297–310, 2018.
A. Paraschos, C. Daniel, J. R. Peters, and G. Neumann, “Probabilistic movement primitives,” Advances in neural information processing systems, vol. 26, 2013.
F. C. Fernandez and W. Caarls, “Deep reinforcement learning for haptic shared control in unknown tasks,” arXiv preprint arXiv:2101.06227, 2021.
V. Havard, B. Jeanne, M. Lacomblez, and D. Baudry, “Digital twin and virtual reality: a co-simulation environment for design and assessment of industrial workstations,” Production & Manufacturing Research, vol. 7, no. 1, pp. 472–489, 2019.
K. Rangarajan, H. Davis, and P. H. Pucher, “Systematic review of virtual haptics in surgical simulation: a valid educational tool?” Journal of surgical education, vol. 77, no. 2, pp. 337–347, 2020.
K. Salisbury, F. Conti, and F. Barbagli, “Haptic rendering: introductory concepts,” IEEE computer graphics and applications, vol. 24, no. 2, pp. 24–32, 2004.
S. Sadeghnejad, F. Khadivar, E. Abdollahi, H. Moradi, F. Farahmand, S. M. Sadr Hosseini, and G. Vossoughi, “A validation study of a virtual-based haptic system for endoscopic sinus surgery training,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 15, no. 6, p. e2039, 2019.
R. L. Williams, M. Srivastava, R. Conaster, and J. N. Howell, “Implementation and evaluation of a haptic playback system,” Haptics-e, vol. 3, no. 3, 2004.
R. Varshney, S. Frenkiel, L. H. Nguyen, M. Young, R. Del Maestro, A. Zeitouni, E. Saad, W. R. J. Funnell, and M. A. Tewfik, “The mcgill simulator for endoscopic sinus surgery (msess): a validation study,” Journal of Otolaryngology-Head & Neck Surgery, vol. 43, no. 1, pp. 1–10, 2014.
W. C. Agboh, M. Yalcin, and V. Patoglu, “A six degrees of freedom haptic interface for laparoscopic training,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp. 1107–1112.
H. Cakmak, H. Maass, and U. K ̈uhnapfel, “Vs one, a virtual reality simulator for laparoscopic surgery,” Minimally Invasive Therapy & Allied Technologies, vol. 14, no. 3, pp. 134–144, 2005.
C. V ̊apenstad, E. F. Hofstad, L. E. Bø, M. K. Chmarra, E. Kuhry, G. Johnsen, R. M ̊arvik, and T. Langø, “Limitations of haptic feedback devices on construct validity of the lapsim® virtual reality simulator,” Surgical endoscopy, vol. 27, pp. 1386–1396, 2013.
S. Misra, K. Ramesh, and A. M. Okamura, “Modelling of non-linear elastic tissues for surgical simulation,” Computer methods in biomechanics and biomedical engineering, vol. 13, no. 6, pp. 811–818, 2010.
J. Zhang, Y. Zhong, and C. Gu, “Deformable models for surgical simulation: a survey,” IEEE reviews in biomedical engineering, vol. 11, pp. 143–164, 2017.
S. Sadeghnejad, F. Farahmand, G. Vossoughi, H. Moradi, and S. M. S. Hosseini, “Phenomenological tissue fracture modeling for an endoscopic sinus and skull base surgery training system based on experimental data,” Medical engineering & physics, vol. 68, pp. 85–93, 2019.
A. Talasaz, A. L. Trejos, and R. V. Patel, “The role of direct and visual force feedback in suturing using a 7-dof dual-arm teleoperated system,” IEEE transactions on haptics, vol. 10, no. 2, pp. 276–287, 2016.
A. Abiri, S. J. Askari, A. Tao, Y.-Y. Juo, Y. Dai, J. Pensa, R. Candler, E. P. Dutson, and W. S. Grundfest, “Suture breakage warning system for robotic surgery,” IEEE Transactions on Biomedical Engineering, vol. 66, no. 4, pp. 1165–1171, 2018.
N. Haouchine, W. Kuang, S. Cotin, and M. Yip, “Vision-based force feedback estimation for robot-assisted surgery using instrumentconstrained biomechanical three-dimensional maps,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2160–2165, 2018.
A. Saracino, A. Deguet, F. Staderini, M. N. Boushaki, F. Cianchi, A. Menciassi, and E. Sinibaldi, “Haptic feedback in the da vinci research kit (dvrk): A user study based on grasping, palpation, and incision tasks,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 15, no. 4, p. e1999, 2019.
C. Pacchierotti, D. Prattichizzo, and K. J. Kuchenbecker, “Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 2, pp. 278–287, 2015.
J. K. Koehn and K. J. Kuchenbecker, “Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery,” Surgical endoscopy, vol. 29, pp. 2970–2983, 2015.
A. J. Spiers, H. J. Thompson, and A. G. Pipe, “Investigating remote sensor placement for practical haptic sensing with endowrist surgical tools,” in 2015 IEEE World Haptics Conference (WHC). IEEE, 2015, pp. 152–157.
E. S. Toberer, L. L. Baranowski, and C. Dames, “Advances in thermal conductivity,” Annual Review of Materials Research, vol. 42, pp. 179–209, 2012.
A. G. Gallagher and G. C. O’Sullivan, Fundamentals of surgical simulation: principles and practice. Springer Science & Business Media, 2011.
E. N. Spruit, G. P. Band, J. F. Hamming, and K. R. Ridderinkhof, “Optimal training design for procedural motor skills: a review and application to laparoscopic surgery,” Psychological research, vol. 78, pp. 878–891, 2014.
E. M. Ritter and D. J. Scott, “Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery,” Surgical innovation, vol. 14, no. 2, pp. 107–112, 2007.
R. C. Wilson, A. Shenhav, M. Straccia, and J. D. Cohen, “The eighty five percent rule for optimal learning,” Nature communications, vol. 10, no. 1, p. 4646, 2019.
S. G. Hart and L. E. Staveland, “Development of nasa-tlx (task load index): Results of empirical and theoretical research,” in Advances in psychology. Elsevier, 1988, vol. 52, pp. 139–183.
G. Fenza, F. Orciuoli, and D. G. Sampson, “Building adaptive tutoring model using artificial neural networks and reinforcement learning,” in 2017 IEEE 17th international conference on advanced learning technologies (ICALT). IEEE, 2017, pp. 460–462.
T. Minematsu, A. Shimada, and R.-i. Taniguchi, “Simple background subtraction constraint for weakly supervised background subtraction network,” in 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 2019, pp. 1–8.
M. Hong, K. Meisner, S. Lee, A. M. Schreiber, and J. W. Rozenblit, “A fuzzy reasoning system for computer-guided laparoscopy training,” in 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2020, pp. 1712–1717.
Additional Files
Published
Issue
Section
License
Copyright (c) 2024 International Journal of Electronics and Telecommunications
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
1. License
The non-commercial use of the article will be governed by the Creative Commons Attribution license as currently displayed on https://creativecommons.org/licenses/by/4.0/.
2. Author’s Warranties
The author warrants that the article is original, written by stated author/s, has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author/s. The undersigned also warrants that the manuscript (or its essential substance) has not been published other than as an abstract or doctorate thesis and has not been submitted for consideration elsewhere, for print, electronic or digital publication.
3. User Rights
Under the Creative Commons Attribution license, the author(s) and users are free to share (copy, distribute and transmit the contribution) under the following conditions: 1. they must attribute the contribution in the manner specified by the author or licensor, 2. they may alter, transform, or build upon this work, 3. they may use this contribution for commercial purposes.
4. Rights of Authors
Authors retain the following rights:
- copyright, and other proprietary rights relating to the article, such as patent rights,
- the right to use the substance of the article in own future works, including lectures and books,
- the right to reproduce the article for own purposes, provided the copies are not offered for sale,
- the right to self-archive the article
- the right to supervision over the integrity of the content of the work and its fair use.
5. Co-Authorship
If the article was prepared jointly with other authors, the signatory of this form warrants that he/she has been authorized by all co-authors to sign this agreement on their behalf, and agrees to inform his/her co-authors of the terms of this agreement.
6. Termination
This agreement can be terminated by the author or the Journal Owner upon two months’ notice where the other party has materially breached this agreement and failed to remedy such breach within a month of being given the terminating party’s notice requesting such breach to be remedied. No breach or violation of this agreement will cause this agreement or any license granted in it to terminate automatically or affect the definition of the Journal Owner. The author and the Journal Owner may agree to terminate this agreement at any time. This agreement or any license granted in it cannot be terminated otherwise than in accordance with this section 6. This License shall remain in effect throughout the term of copyright in the Work and may not be revoked without the express written consent of both parties.
7. Royalties
This agreement entitles the author to no royalties or other fees. To such extent as legally permissible, the author waives his or her right to collect royalties relative to the article in respect of any use of the article by the Journal Owner or its sublicensee.
8. Miscellaneous
The Journal Owner will publish the article (or have it published) in the Journal if the article’s editorial process is successfully completed and the Journal Owner or its sublicensee has become obligated to have the article published. Where such obligation depends on the payment of a fee, it shall not be deemed to exist until such time as that fee is paid. The Journal Owner may conform the article to a style of punctuation, spelling, capitalization and usage that it deems appropriate. The Journal Owner will be allowed to sublicense the rights that are licensed to it under this agreement. This agreement will be governed by the laws of Poland.
By signing this License, Author(s) warrant(s) that they have the full power to enter into this agreement. This License shall remain in effect throughout the term of copyright in the Work and may not be revoked without the express written consent of both parties.
