Augmented reality based brain tumor 3D visualization

In this paper we present an augmented reality system for mobile devices that facilitates 3D brain tumor visualization in real time. The system uses facial features to track the subject in the scene. The system performs camera calibration based on the face size of the subject, instead of the common approach of using a number of chessboard images to calibrate the camera every time the application is installed on a new device. Camera 3D pose estimation is performed by finding its position and orientation based on a set of 3D points and their corresponding 2D projections. According to the estimated camera pose, a reconstructed brain tumor model is displayed at the same location as the subject’s real anatomy. The results of our experiment show the system was successful in performing the brain tumor augmentation in real time with a reprojection accuracy of 97%.