Diagnostic and Personalized Skin Care via Artificial Intelligence (invited talk)

PARC is hosting a special session on  “Computer Vision and Artificial Intelligence for Health & Beauty Applications session” at the 2019 IS&T Electronic Imaging Symposium in San Francisco on January 16th. The session, chaired by PARC Research Scientist Raja Bala, will comprise talks by leading researchers from PARC, Proctor & Gamble, L’Oreal, and Stanford University. Kicking off the session is an invited talk by PARC Research Scientist Matt Shreve and Procter & Gamble Principal Scientist, Beauty Next Group, Ankur Purwar. The mass beauty aisle is often crowded and confusing and experiences  with beauty counselors in specialty department stores can be overwhelming. In both cases, women are walking away less than satisfied with their shopping experience and not certain that their products are exactly right for their skin. Part of this  dissatisfaction stems from the fact that women want personalized  attention and recommendations, and the expectation of what “personalized” means has changed. In this talk, we will share an  overview of how Computer Vision and Artificial Intelligence (AI) solutions are being leveraged to diagnose and personalize beauty care.  In the first part of the talk, we will provide the background, motivation and impact for the newly developed Olay Skin Advisor, a web-based skin analyst application and advisor tool, that is  leveraging AI to create personalized beauty experiences to address this problem.  In the second part, we will describe a mobile application that is being developed to assess a consumer’s perceived “skin age” from a smartphone selfie image. A novel aspect of this application is the real-time computation of quality measures that enable the reliable capture of “scientific selfies”, i.e., selfies with sufficient image quality so that meaningful measurements can be extracted. Example quality measures include a range of traditional checks such as illumination, capture distance and focus, as well as deep-learning based measures that detect attributes such as facial expression, hair or other facial occlusions, and pose, among others.  We will then present the main application that predicts a user’s skin age across various regions of the face using deep learning. The method works by extracting several skin patches in specific regions of the face, and trains a convolutional neural network (CNN) on each region separately. Each regional CNN model is then fine-tuned using a novel data augmentation technique that artificially reduces the apparent age of the skin through a series of smoothing operations that act as a proxy for subjects with younger looking skin. The deep features extracted from each of these regions are then used to train a separate set of regression models that predict a user's skin age.