top of page


Real-to-virtual domain transfer-based depth estimation for real-time 3D annotation in transnasal surgery: a study of annotation accuracy and stability


Mr. Tong Hon Sing

MPhil candidate in the Mechanical Engineering Dept.

Date & Time

Tuesday, 27 April 2021

7:00 am


Via Zoom


Surgical annotation promotes effective communication between medical personnel during surgical procedures. However, existing approaches to 2D annotations are mostly static with respect to a display. In this study, we propose a method to achieve 3D annotations that anchor rigidly and stably to target structures upon camera movement in a transnasal endoscopic surgery setting. This is accomplished through intra-operative endoscope tracking and monocular depth estimation. In total, 2 deep neural networks are involved in monocular depth estimation. The first one is an adversarial network that transfers image style from the real endoscopic view to a synthetic-like view, while the second one is a fully-supervised depth estimation network that is trained in a virtual endoscopic environment prior to the prediction phase. Accuracy and stability evaluations are performed on an anatomically-accurate nasal airway phantom to demonstrate the feasibility and practicality of our proposed method for achieving 3D annotations.

Research Areas:

Robotics and Control

Contact for


Dr. K.W. Kwok

+(852) 3917 2636

Download Details

in PDF

bottom of page