Modifying Digital Content With Digital Effects Using Facial Skin Mask

Patent metadata

Field	Value
Publication number	US-2020175736-A1
Application number	US-201816205010-A
Country	US
Kind code	A1
Filing date	Nov 29, 2018
Priority date	Nov 29, 2018
Publication date	Jun 4, 2020
Grant date	—

Abstract

Facial skin mask generated by a digital content creation system is described. The digital content creation system includes digital effects on skin in facial regions of digital content with efficiency and accuracy. Upon identifying a facial region within digital content, the system generates a first regional skin mask, a second regional skin mask, and combines both of the first and second regional skin masks to generate a facial skin mask indicative of skin of the identified facial regions depicted in digital content. The digital content creation system then modifies digital content by applying user selected digital effects to the skin of the facial region using the generated facial skin mask.

First claim

1 . In a digital medium environment, a method implemented by at least one computing device to apply a digital effect to digital content, the method comprising:
identifying, by the at least one computing device, a facial region in the digital content; generating, by the at least one computing device, a first regional skin mask on a first region of the facial region and at least a second regional skin mask on a second region of the facial region; generating, by the at least one computing device, a facial skin mask by combining the first and second regional skin masks, the facial skin mask indicative of skin of the facial region; receiving, by the at least one computing device, a user selection of the digital effect to be applied to the facial region; and modifying, by the at least one computing device, the digital content by applying the digital effect on the skin of the facial region using the facial skin mask.

2 . The method as described in claim 1 , wherein the digital effect comprises at least one of digital paint, digital skin tone, digital lighting, or digital smoothing

3 . The method as described in claim 1 , wherein the facial region is defined by landmarks.

4 . The method as described in claim 1 , wherein generating the first regional skin mask comprises:
extracting a subset of landmarks from landmarks defining the facial region, the extracted subset of landmarks defining the first region of the facial region, the first region corresponding to a forehead comprising hair interspersed with skin; and segmenting the first region into a first segment corresponding to the hair and a second segment corresponding to the skin.

5 . The method as described in claim 4 , wherein segmenting the first region into the first and second segments comprises:
identifying pixels associated with the subset of landmarks defining the first region; and processing the pixels associated with the subset of landmarks defining the first region using k-means algorithm such that the pixels corresponding to the hair of the forehead and the pixels corresponding to the skin of the forehead are separated into the first segment and the second segment, respectively.

6 . The method as described in claim 4 , wherein applying the digital effect to the skin of the facial region is effective to apply the digital effect to the second segment corresponding to the skin of the forehead.

7 . The method as described in claim 1 , wherein generating the second regional skin mask comprises extracting a subset of landmarks from landmarks defining the facial region, the extracted subset of landmarks defining the second region of the facial region.

8 . The method as described in claim 7 , further comprising:
tracing contours of the second region of the facial region based on the subset of landmarks defining the second region of the facial region, the contours used to identify portions of the facial region that correspond to the skin and portions of the facial region that correspond to excluded facial features.

9 . The method as described in claim 8 , wherein applying the digital effect to the skin of the facial region using the facial skin mask is effective to apply the digital effect to the portions of the facial region that correspond to the skin without applying the digital effect to the portions of the facial region that correspond to the excluded facial features.

10 . The method as described in claim 1 , further comprising:
detecting a position change of the facial region within the digital content and comparing the position change to a predetermined threshold; generating another facial skin mask if the position change exceeds the predetermined threshold; and retaining the facial skin mask if the position change is less than or equal to the predetermined threshold.

11 . In a digital medium environment, a system configured to apply a digital effect to digital content, the system comprising:
a facial region identification module implemented at least partially in hardware of a computing device to identify a facial region in the digital content; a first regional skin mask generation module implemented at least partially in hardware of the computing device to generate a first regional skin mask on a first region of the facial region; a second regional skin mask generation module implemented at least partially in hardware of the computing device to generate a second regional skin mask on a second region of the facial region; a facial skin mask generation module implemented at least partially in hardware of the computing device to generate a facial skin mask by combining the first and second regional skin masks, the facial skin mask indicative of skin of the facial region; a digital effect input module implemented at least partially in hardware of the computing device to receive a user selection of the digital effect to be applied to the facial region; and the facial skin mask generation module implemented to modify the digital content by applying the digital effect on the skin of the facial region using the facial skin mask.

12 . The system as described in claim 11 , wherein the digital effect comprises at least one of digital paint, digital skin tone, digital lighting, or digital smoothing

13 . The system as described in claim 11 , wherein the facial region is defined by landmarks.

14 . The system as described in claim 11 , wherein the first regional skin mask generation module generates the first regional skin mask by:
extracting a subset of landmarks from landmarks defining the facial region, the subset of landmarks defining the first region of the facial region, the first region corresponding to a forehead comprising hair interspersed with skin; and segmenting the first region into a first segment corresponding to the hair and a second segment corresponding to the skin.

15 . The system as described in claim 14 , wherein the first regional skin mask generation module segments the first region into the first and second segments by:
identifying pixels associated with the subset of landmarks defining the first region; and processing the pixels associated with the subset of landmarks defining the first region using k-means algorithm such that the pixels corresponding to the hair of the forehead and the pixels corresponding to the skin of the forehead are separated into the first segment and the second segment, respectively.

16 . The system as described in claim 14 , wherein the facial skin mask generation module applying the digital effect to the skin of the facial region is effective to apply the digital effect to the second segment corresponding to the skin of the forehead.

17 . The system as described in claim 11 , wherein the second regional skin mask generation module generates the second regional skin mask by:
extracting a subset of landmarks from landmarks defining the facial region, the subset of landmarks defining the second region of the facial region; and tracing contours of the second region of the facial region based on the subset of landmarks defining the second region of the facial region, the contours used to identify portions of the facial region that correspond to the skin and portions of the facial region that correspond to excluded facial features.

18 . The system as described in claim 17 , wherein the facial skin mask generation module applies the digital effect to portions of the facial region that correspond to the skin without applying the digital effect to the portions of the facial region that correspond to the excluded facial featu

Assignees

• Adobe Inc

Inventors

• Jia Yumin
• Lu Xin
• Chien Jen-Chan

Classifications

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/12

  Edge-based segmentation · CPC title

• G06K9/00234

  Physics · mapped topic

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title