Computer vision using hyperspectral imaging & machine learning to classify tumor, margin, and soft-tissues of the head and neck

Importance: The head and neck surgeon faces many visual tissue discrimination tasks that remain challenging even when aided by optical magnification, such as distinguishing tumor from surrounding normal tissues, nerve from vessel, and parathyroid from fat. The human retina relies upon trichromatic (RGB) color discrimination and cannot perceive differences in the wavelengths of light reflected from heterogeneous human tissues if those differences do not vary the trichromatic stimulus. Objective: We hypothesize that there is clinically valuable information that can be discerned using hyperspectral imaging of animal and human tissues. The purpose of this study was to demonstrate tissue discrimination by applying machine-learning and computer vision algorithms to hyperspectral images of tissue acquired with a novel hyperspectral camera. Materials and Methods: A PARC hyperspectral camera prototype, based on a liquid-crystal polarization interferometer with approx. 500 cm-1 resolution over the visible and near-infrared portion of the electromagnetic spectrum, was used to acquire hyperspectral image datasets of freshly obtained animal and human tissue. The datasets were analyzed with conventional machine-learning methods for clustering and classification. Results: The hyperspectral camera was used to image the soft-tissues of the head and neck from 25 mice. Distinct spectral patterns were discerned with unsupervised clustering and principal components analysis that could be correlated with differences in tissue type. Such statistical analyses could potentially be used to distinguish thyroid gland from submandibular gland, mucosal oral tongue from cervical muscle, and lung from liver. The camera was also used to acquire hyperspectral datacubes of freshly resected oral and oropharyngeal cancer tissue to train a classifier for later discrimination of tumor from normal tissue. Finally, hyperspectral imaging of the human oral cavity was performed in vivo on a human subject using a 7 mm rigid endoscope. Based on differences in the wavelengths of reflected light, a trained classifier successfully discerned between subsites within the oral cavity that nevertheless had a similar visual presentation. (Figures 1 and 2). Conclusions and Relevance: Our study demonstrates the feasibility of using hyperspectral imaging and computer vision to classify and to distinguish clinically significant differences of murine and human soft-tissues. The speed, convenience, and spectral resolution of the PARC hyperspectral camera suggest that this technology might provide the head and neck surgeon with fundamentally enhanced intra-operative surgical vision. Implementation and future study may be warranted in a prospective clinical trial.