Estee Lauder uses AI/AR technology to make makeup more refined, so people with short-sightedness don’t have to worry about not being able to apply makeup properly.

The cosmetics giant has released a voice-enabled makeup assistant application, which uses AI and augmented reality (AR) technology to help visually impaired people easily outline their makeup.

How can we help visually impaired people use Estee Lauder products and enhance their autonomy and confidence? Faced with this real problem, the cosmetics giant decided to find answers from AI and augmented reality (AR) technology.

The result is the voice-enabled makeup assistant (VMA). This application has been fully launched in the UK and the United States, and will soon be launched in the global market. With this project, Estee Lauder successfully won the 2023 CIO 100 IT Excellence Award.

Michael Smith, senior vice president and CIO of The Estée Lauder Companies, said, “We are addressing a major business problem in the beauty industry, which is the severe lack of accessible solutions. Globally There are 2.2 billion people living with some degree of visual impairment, over 2 million in the UK alone, and this will be our trial environment. It should be emphasized that this represents not only a business opportunity, but also is in line with The Estée Lauder Company’s Our mission is to become the most inclusive and diverse beauty company in the world. Whether it is facing internal employees or all consumers, Estée Lauder’s determination will not waver.”

The Estée Lauder Company Innovation and Christopher Aidan, vice president of emerging technologies, added that visually impaired people often need help from others to handle mundane tasks in daily life. To this end, VMA hopes to use Estee Lauder's existing augmented reality immersive application (ARIA) platform to use AR/AI and machine learning (ML) algorithms to analyze users' facial makeup, and then VMA will help users create their ideal makeup through voice guidance. Appearance.

VMA is a mobile app that enables voice control through user pre-configured preferences on the device, while voice and speaking speed can be customized through settings options in Accessibility.

Correct use of AI technology

Although it does not take much time for the technical team led by Aidan to build such an application, there is a first step they need to take. The hard part is completing the feasibility study first. As a technology project with inclusivity as its guiding principle, the team realized that the app would need to adapt to the various facial features of any potential user, including face shape, size, skin tone, and other unique features. In order to ensure that the AI ​​can adapt to these appearance factors and respond correctly in combination with other diversity indicators such as hairstyle, the training process of the model is far more complicated than initially expected. In other words, they need to ensure that the training corpus has wider diversity.

The original design hopes to capture selfies, then use analysis algorithms to evaluate application consistency, and then provide users with makeup guidance. But it didn’t take long for the team to discover that the app could scan a user’s face and generate a live video. For example, if a video shows that the user's foundation or lipstick is not evenly applied, the app will provide verbal descriptions and guidance on the specific areas that need to be retouched to help visually impaired users understand what to do next. When the makeup is perfect and flawless, the app will also issue a prompt to inform users that no further adjustments are needed.

In addition, Aidan and his team’s top priority is to maintain direct contact with the visually impaired community. “We really wanted to understand what their unique needs were, what their pain points were, what their preferences were, and what they wanted from Estée Lauder products. We convened the trial group and asked them questions and listened carefully to what they thought about makeup. and personal experience at the technical level."

More importantly, some members of the experimental group are completely blind, some members have weak vision, and some members still have intact peripheral vision. Through contact and communication with them, the Aidan team was able to compile first-hand experiences and insights, and question assumptions they originally thought were reliable.

Aidan said, "We originally thought that a more natural and more personalized voice would definitely gain more support from everyone, but user surveys proved that sounding familiar is the most important factor. In other words, they want Hearing a voice that is the same/similar to the preset tone on your phone is like having the same old friend.” The team collaborated with other teams to combine these insights with feedback from the trial team to establish VMA development ideas. After that, the team will also combine user research to complete all stages of work from application naming to voice/tone design.

Aidan explained, “Throughout the design, build and testing phases, feedback from the visually impaired community informed our decision-making, allowing us to even address small issues such as adjusting the speaking speed of our virtual assistant. Get huge inspiration."

Still a work in progress

The team has been continuously monitoring market feedback on the app since it was first launched in the UK in January this year. Aidan noted that despite extensive testing before release, users were finding new issues with it.

“We found that many of the questions raised by users were indeed not even thought of in early testing. For example, they would ask “What do I do if I want to remove the app? How can I identify myself?” Has it been successfully deleted?"

Smith explained that Estee Lauder is measuring the success of the VMA through user feedback. So far, market feedback is quite positive and constructive.

Smith said, “Some users mentioned, ‘This is a great app that I like so much that I can’t even imagine how to live without it.’” This software makes users feel strongly empowered, and they can Try more new products, be able to ask questions decisively, and no longer dwell on the negative feelings of having to rely on others. More importantly, this app does not make any judgments, but simply objectively restores reality. "

Smith also pointed out that the VMA application can help more than just the visually impaired. Young users who lack adult guidance in their lives or are unwilling to ask adults for help can also use this convenient application Understand and learn how to design makeup.

He concluded, “I would challenge other CIOs to see who can create products that are easier to use and more inclusive. Only this attitude of always considering inclusivity at the level of product functionality and audience can create results that benefit everyone. ”

The above is the detailed content of Estee Lauder uses AI/AR technology to make makeup more refined, so people with short-sightedness don’t have to worry about not being able to apply makeup properly.. For more information, please follow other related articles on the PHP Chinese website!