Enabling Smarter Skin Care with Machine Learning

OVERVIEW

Olay, the Procter & Gamble (P&G) skin care leader, is committed to the science of skin care. Olay collaborated with PARC to develop a deep learning-powered skin analysis and recommendation platform to enable smarter, more personalized skin care for women. 

OBJECTIVE

Develop a platform that enables smarter skin care choices
Making the right skin care choices can be difficult. For many women today, an overabundance of products on the market, together with a lack of deep scientific skin care knowledge, can result in uninformed experimentation with skin care products. This can lead to frustration, wasted money, and undesirable results. Olay wanted to develop an easy-to-use platform that would make skin care smarter and more personalized for everyday consumers.

WHY PARC?

Deep capabilities in building advanced machine learning systems and user experience design
How do you take the knowledge found inside a lab or behind a counter to personalize product selection for every woman’s unique face? The answer is machine learning. Machine learning uses algorithms to learn from and make predictions from large amounts of data, in this case, skin features. 

Olay partnered with PARC because of their deep experience and capabilities in building advanced machine learning systems across industries. Using PARC’s user experience design capability, PARC and Olay could co-develop the right platform to effectively communicate that skin care knowledge and create a sustained relationship with its users. 

DESIGN-LED, RIGOROUS EXPLORATION OF ACCESSIBLE MARKETS (DREAM)

MULTI-PHASED USER EXPERIENCE RESEARCH

User experience probes
To address complex audience needs, new business models, and emerging technologies, PARC developed an iterative, hypothesis-driven research technique to explore the value of skin care advice solutions based around sensors and selfies.

PARC interviewed and observed Olay’s target consumers to better understand:

• Attitudes toward personal appearance and skin care 
• How and when users took and shared selfies
• User expectations and reactions around a skin care advice solution
• How to foster user trust when presenting advice with the intent to sell

PARC worked with Olay to co-construct a strategic roadmap for creating a technology-based solution to address the unique needs of the customer.

Technology development
Working closely with PARC’s user experience design team, PARC’s computer vision scientists developed software to guide the user to capture selfies under optimal conditions by controlling variables such as lighting, camera distance and facial expression. 

PARC’s machine learning scientists:

• Developed deep learning algorithms to make predictions of skin condition from selfies
• Trained models to detect the presence of target skin features, using human-graded image databases as ground truth

To ensure that the deep learning algorithms could robustly classify what was most valuable to consumers, PARC worked with Olay to design the annotation requirements for the image datasets that were being built.

Prototype design
PARC designed and built increasingly higher fidelity “looks-like” and “works-like” prototypes for Olay, enabling further user validation to occur while the platform was being developed. Prototypes were shown to Olay customers to evaluate their overall experience. 

To optimize the user experience and minimize disengagement, PARC:

• Designed an interface that guides customers through taking a selfie to maximize the quality of input data for deep learning analysis (rather than simply being a good-looking selfie to the user)
• Explored the optimal way to deliver key information back to the user with personalized, easy to understand recommendations

SOLUTION

Olay Skin Advisor machine learning-powered platform
After an initial round of user studies and experience probes, PARC and Olay agreed to start with a web-based platform with three forms of artificial intelligence—computer vision, deep learning, and adaptive recommendations—that could deliver personalized skin diagnosis using selfies, and recommend individualized product and regimen changes. Repeated use would improve results and provide deeper insights, as well as deepen the sustained relationship between each user and Olay. 

RESULTS

Over 4 million site visits and continued co-development
PARC delivered to Olay a suite of algorithms and user experience elements and, in September of 2016, the Olay Skin Advisor platform was released. The platform enabled accurate analysis of users’ skin, informed users of what was happening with their skin, suggested product and regimen changes, and provided a compelling user interaction flow.

Since launch, the platform has reached several notable milestones for Olay, including: 

• Over 4 million site visits worldwide 
• Users of the platform exhibited 2x the conversion rate and 40% larger basket size upon checkout as compared to regular Olay.com visitors. They also yielded 3x lower bounce rate and 4x time spent as engagement measures.
• Local versions in ten countries

Today, PARC and Olay continue to explore how to grow and expand the feature set of the Olay Skin Advisor platform, extending the strategic roadmap established at the start of the partnership. 

Develop a platform that enables smarter skin care choices

Making the right skin care choices can be difficult. For many women today, an overabundance of products on the market, together with a lack of deep scientific skin care knowledge, can result in uninformed experimentation with skin care products. This can lead to frustration, wasted money, and undesirable results. Olay wanted to develop an easy-to-use platform that would make skin care smarter and more personalized for everyday consumers.

Deep capabilities in building advanced machine-learning systems and user experience design

How do you take the knowledge found inside a lab or behind a counter to personalize product selection for every woman’s unique face? The answer is machine learning. Machine learning uses algorithms to learn from and make predictions from large amounts of data, in this case, skin features.

Olay partnered with PARC because of their deep experience and capabilities in building advanced machine learning systems across industries. Using PARC’s user experience design capability, PARC and Olay could co-develop the right platform to effectively communicate that skin care knowledge and create a sustained relationship with its users.

DESIGN-LED, RIGOROUS EXPLORATION OF ACCESSIBLE MARKETS (DREAM)

User experience probes

To address complex audience needs, new business models, and emerging technologies, PARC developed an iterative, hypothesis-driven research technique to explore the value of skin care advice solutions based around sensors and selfies.

PARC interviewed and observed Olay’s target consumers to better understand:

• Attitudes toward personal appearance and skin care
• How and when users took and shared selfies
• User expectations and reactions around a skin care advice solution
• How to foster user trust when presenting advice with the intent to sell

PARC worked with Olay to co-construct a strategic roadmap for creating a technology-based solution to address the unique needs of the customer.

Technology development

Working closely with PARC’s user experience design team, PARC’s computer vision scientists developed software to guide the user to capture selfies under optimal conditions by controlling variables such as lighting, camera distance and facial expression.

PARC’s machine learning scientists:

• Developed deep learning algorithms to make predictions of skin condition from selfies
• Trained models to detect the presence of target skin features, using human-graded image databases as ground truth

To ensure that the deep learning algorithms could robustly classify what was most valuable to consumers, PARC worked with Olay to design the annotation requirements for the image datasets that were being built.

Prototype design

PARC designed and built increasingly higher fidelity “looks-like” and “works-like” prototypes for Olay, enabling further user validation to occur while the platform was being developed. Prototypes were shown to Olay customers to evaluate their overall experience.

To optimize the user experience and minimize disengagement, PARC:

• Designed an interface that guides customers through taking a selfie to maximize the quality of input data for deep learning analysis (rather than simply being a good-looking selfie to the user)
• Explored the optimal way to deliver key information back to the user with personalized, easy to understand recommendations

Olay Skin Advisor machine learning-powered platform

After an initial round of user studies and experience probes, PARC and Olay agreed to start with a web-based platform with three forms of artificial intelligence—computer vision, deep learning, and adaptive recommendations—that could deliver personalized skin diagnosis using selfies, and recommend individualized product and regimen changes. Repeated use would improve results and provide deeper insights, as well as deepen the sustained relationship between each user and Olay.

Over 4 million site visits and continued co-development

PARC delivered to Olay a suite of algorithms and user experience elements and, in September of 2016, the Olay Skin Advisor platform was released. The platform enabled accurate analysis of users’ skin, informed users of what was happening with their skin, suggested product and regimen changes, and provided a compelling user interaction flow.

Since launch, the platform has reached several notable milestones for Olay, including:

• Over 4 million site visits worldwide
• Users of the platform exhibited 2x the conversion rate and 40% larger basket size upon checkout as compared to regular Olay.com visitors. They also yielded 3x lower bounce rate and 4x time spent as engagement measures.
• Local versions in ten countries

Today, PARC and Olay continue to explore how to grow and expand the feature set of the Olay Skin Advisor platform, extending the strategic roadmap established at the start of the partnership.