Augmented Reality During Robot-assisted Laparoscopic Partial Nephrectomy: Toward Real-Time 3D-CT to Stereoscopic Video Registration
Received 20 August 2008; accepted 23 November 2008. published online 04 February 2009.
Objectives
To investigate a markerless tracking system for real-time stereo-endoscopic visualization of preoperative computed tomographic imaging as an augmented display during robot-assisted laparoscopic partial nephrectomy.
Methods
Stereoscopic video segments of a patient undergoing robot-assisted laparoscopic partial nephrectomy for tumor and another for a partial staghorn renal calculus were processed to evaluate the performance of a three-dimensional (3D)-to-3D registration algorithm. After both cases, we registered a segment of the video recording to the corresponding preoperative 3D-computed tomography image. After calibrating the camera and overlay, 3D-to-3D registration was created between the model and the surgical recording using a modified iterative closest point technique. Image-based tracking technology tracked selected fixed points on the kidney surface to augment the image-to-model registration.
Results
Our investigation has demonstrated that we can identify and track the kidney surface in real time when applied to intraoperative video recordings and overlay the 3D models of the kidney, tumor (or stone), and collecting system semitransparently. Using a basic computer research platform, we achieved an update rate of 10 Hz and an overlay latency of 4 frames. The accuracy of the 3D registration was 1 mm.
Conclusions
Augmented reality overlay of reconstructed 3D-computed tomography images onto real-time stereo video footage is possible using iterative closest point and image-based surface tracking technology that does not use external navigation tracking systems or preplaced surface markers. Additional studies are needed to assess the precision and to achieve fully automated registration and display for intraoperative use.
aDepartment of Urology, University of Florida College of Medicine, Gainesville, Florida
bEngineering Research Center, Computer Integrated Surgical Systems and Technology, Johns Hopkins University, Baltimore, Maryland
Reprint requests: Li-Ming Su, M.D., Department of Urology, University of Florida College of Medicine, 1600 SW Archer Road, P.O. Box 100247, Gainesville, FL 32610
This study was supported in part by a U.S. Army Medical Research and Material Command contract (W81XWH-06-1-0195); a National Science Foundation Engineering Research Center cooperative agreement (EEC9731478), and Johns Hopkins University internal funds.
ABSTRACT