Augmented reality display method of applying color of hair to eyebrows

Patent metadata

Field	Value
Publication number	US-10964070-B2
Application number	US-201916689176-A
Country	US
Kind code	B2
Filing date	Nov 20, 2019
Priority date	May 10, 2019
Publication date	Mar 30, 2021
Grant date	Mar 30, 2021

Abstract

An augmented reality display method of applying color of hair to eyebrows includes the steps of capturing a head image (30) of a user (2), retrieving an original eyebrow image from the head image (30), extracting a hair representation image (31) covering at least part of the user's hair from the head image (30), executing a process of extracting hair region on the hair representation image (31) for obtaining a hair mask for indicating hair's position, computing a hair color parameter according to the hair mask and the hair representation image (31), executing a process of coloring eyebrows on the head image (30) according to the hair color parameter and position of the original eyebrow image for obtaining an AR (Augmented Reality) head image which its eyebrow color corresponds to the hair color parameter, and displaying the AR head image.

First claim

What is claimed is:

1. An augmented reality display method of applying color of hair to eyebrows, the method being applied to a smart mirror apparatus ( 1 ) comprising an image capture module ( 12 ), a display module ( 11 ) and a processing unit ( 10 ), the method comprising following steps:
a) capturing a head image ( 30 ) of a user ( 2 ) by the image capture module ( 12 );
b) cropping the head image ( 30 ) for obtaining an original eyebrow image by the processing unit ( 10 );
c) cropping the head image ( 30 ) for obtaining a hair representation image ( 31 ), wherein the hair representation image ( 31 ) only covers partial hair of the user ( 2 );
d) executing a process of extracting hair region on the hair representation image ( 31 ) for obtaining a hair mask, wherein the hair mask indicates a hair position of the hair in the hair representation image ( 31 );
e) computing a hair color parameter according to the hair mask and the hair representation image ( 31 );
f) executing a process of coloring eyebrows on the head image ( 30 ) according to the hair color parameter and a position of the original eyebrow image for obtaining an AR (Augmented Reality) head image, wherein an eyebrow color of the AR head image corresponds to the hair color parameter; and
g) displaying the AR head image by the display module ( 11 );
wherein the step d) comprises following steps:
d1) executing a gray-scale process on the hair representation image ( 31 ) to obtain the hair representation image ( 31 ) being gray-scale, and executing an analysis of hairline features on the hair representation image ( 31 ) being gray-scale to obtain a hairline feature image; and
d2) executing a binarization process of brightness division on the hairline feature image to obtain the hairline feature image being halftone, and executing a process of image morphology on the hairline feature image being halftone to obtain the hair mask, wherein the hairline feature image being halftone is used to indicate a region matching with a designated brightness range in the hairline feature image.

2. The augmented reality display method of applying color of hair to eyebrows according to claim 1 , further comprising a step h) performed after the step a) and before the step b) executing a face analysis process on the head image ( 30 ) to recognize a plurality of eyebrow feature points ( 40 , 41 ) corresponding to eyebrows of the user ( 2 );
wherein the step b) is performed to crop the eyebrow image from the head image ( 30 ) according to the plurality of eyebrow feature points ( 40 , 41 ).

3. The augmented reality display method of applying color of hair to eyebrows according to claim 2 , wherein the step h) is further performed to recognize a plurality of forehead feature points ( 42 - 44 ) corresponding to a forehead of the user ( 2 ), the step c) comprises following steps:
c1) determining a cropping width (W) according to the plurality of forehead feature points ( 42 - 44 );
c2) determining a cropping height (H) according to the forehead feature point ( 42 - 44 ) being the highest and a hair apex point; and
c3) cropping a part of the head image ( 30 ) as the hair representation image ( 31 ) according to a position of the forehead feature point ( 42 - 44 ) being the highest, the cropping width (W) and the cropping height (H).

4. The augmented reality display method of applying color of hair to eyebrows according to claim 1 , wherein the step e) is performed to apply the hair mask to the hair representation image ( 31 ) to obtain the hair representation image ( 31 ) excluding from non-hair image, and compute a mean of pixel values of a plurality of pixels of the hair representation image ( 31 ) being applied as the hair color parameter.

5. The augmented reality display method of applying color of hair to eyebrows according to claim 1 , wherein the step f) further comprises following steps:
f1) retrieving an eyebrow mask corresponding to the original eyebrow image, wherein the eyebrow mask is used to indicate a position and shape of the original eyebrow image in the head image ( 30 );
f2) executing a process of erasing or brushing eyebrows on the head image ( 30 ) to erase or conceal the original eyebrow image; and
f3) executing a first color-mixing process on the head image ( 30 ) being processed according to the hair color parameter and the eyebrow mask to color at an eyebrow position of the head image ( 30 ) being processed to obtain the AR head image.

6. The augmented reality display method of applying color of hair to eyebrows according to claim 5 , wherein the first color-mixing process comprises computing a plurality of new pixel values of the plurality of pixels of the head image ( 30 ) according to a first transparency amount, the hair color parameter and a plurality of original pixel values of the plurality of pixels of the head image ( 30 ) being processed corresponding to the eyebrow mask.

7. The augmented reality display method of applying color of hair to eyebrows according to claim 1 , further comprising a step i) determining a color code corresponding to the hair color parameter, and display the color code by the display module ( 11 ).

8. An augmented reality display method of applying color of hair to eyebrows, the method being applied to a smart mirror apparatus ( 1 ) comprising an image capture module ( 12 ), a display module ( 11 ) and a processing unit ( 10 ), the method comprising following steps:
a) capturing a head image ( 30 ) of a user ( 2 ) by the image capture module ( 12 );
b) cropping the head image ( 30 ) for obtaining an original eyebrow image by the processing unit ( 10 );
c) cropping the head image ( 30 ) for obtaining a hair representation image ( 31 ), wherein the hair representation image ( 31 ) only covers partial hair of the user ( 2 );
d) executing a process of extracting hair region on the hair representation image ( 31 ) for obtaining a hair mask, wherein the hair mask indicates a hair position of the hair in the hair representation image ( 31 );
e) computing a hair color parameter according to the hair mask and the hair representation image ( 31 );
f) executing a process of coloring eyebrows on the head image ( 30 ) according to the hair color parameter and a position of the original eyebrow image for obtaining an AR (Augmented Reality) head image, wherein an eyebrow color of the AR head image corresponds to the hair color parameter; and
g) displaying the AR head image by the display module ( 11 );
wherein the step d) comprises following steps:
d1) executing an analysis of hairline features on the hair representation image ( 31 ) to obtain a hairline feature image, wherein the analysis of hairline features comprises analyzing hairline features by Hessian matrix; and
d2) execute a process of image morphology on the hairline feature image to obtain the hair mask, wherein the process of image morphology comprises executing an erosion process and a dilation process on a plurality of adjacent pixels of the hairline feature image at least two times.

9. The augmented reality display method of applying color of hair to eyebrows according to claim 8 , further comprising a step h) performed after the step a) and before the step b) executing a face analysis process on the head image ( 30 ) to recognize a plurality of eyebrow feature points ( 40 , 41 ) corresponding to eyebrows of the user ( 2 );
wherein the step b) is performed to crop the eyebrow image from the head image ( 30 ) according to the plurality of eyebrow feature points ( 40 , 41 ).

10. The augmented reality display method of applying color of hair to eyebrows according to claim 9 , wherein the step h) is further performed to recognize a plurality of

Assignees

• Cal Comp Big Data Inc

Inventors

• Lin Yung-Hsuan

Classifications

• A45D44/005 Primary

  for selecting or displaying personal cosmetic colours or hairstyle · CPC title

• G06V10/34

  Smoothing or thinning of the pattern; Morphological operations; Skeletonisation · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V40/171 Primary

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title