Control method, controller, smart mirror, and computer readable storage medium

Patent metadata

Field	Value
Publication number	US-2019130652-A1
Application number	US-201816234174-A
Country	US
Kind code	A1
Filing date	Dec 27, 2018
Priority date	Jun 12, 2017
Publication date	May 2, 2019
Grant date	—

Abstract

Embodiments of the present disclosure provide a control method, configured to control a smart mirror ( 100 ). The smart mirror ( 100 ) comprises a camera ( 20 ). The control method comprises: controlling the camera ( 20 ) to capture a current user; determining whether the current user is a registered user; enabling the current user to log into the smart mirror ( 100 ) when the current user is a registered user; controlling the smart mirror ( 100 ) to generate interaction with the current user and output interactive information according to an input of the current user. Embodiments of the present disclosure also provide a controller ( 10 ), a smart mirror ( 100 ) and a computer readable storage medium.

First claim

What is claimed is:

1 . A method for controlling a smart mirror, the smart mirror comprising a camera, the method comprising:
controlling the camera to capture an image of a current user; determining whether the current user is a registered user of the smart mirror; enabling the current user to log into the smart mirror when the current user is a registered user; and controlling the smart mirror to generate interaction with the current user and output interactive information according to an input of the current user.

2 . The control method according to claim 1 , wherein the smart mirror comprises a registry, the registry comprises registered feature information of registered face areas of all the registered users, the step of determining whether the current user is a registered user comprises:
processing a first image of the current user captured by the camera to obtain a face area to be tested of the current user; processing the face area to be tested to obtain feature points to be tested of the face area to be tested; processing the feature points to be tested to extract feature information of the face area to be tested; comparing the feature information to be tested with the registered feature information to obtain a comparison result; and determining that the current user is a registered user when the comparison result is greater than a predetermined threshold.

3 . The control method according to claim 2 , further comprising:
controlling the camera to capture a second image of the current user; and controlling the smart mirror to display the second image.

4 . The control method according to claim 3 , wherein the interaction comprises cuteness processing on the second image, the smart mirror comprises a cuteness material library and the step of controlling the smart mirror to generate interaction with the current user and output interactive information according to the input of the current user comprises:
processing the second image to obtain a cuteness face area of the current user; processing the cuteness face area to obtain cuteness feature points of the cuteness face area; determining cuteness material according to the input of the current user; and matching and fusing the cuteness material with the second image according to the cuteness feature points, to obtain a cuteness face image.

5 . The control method according to claim 3 , wherein the interaction comprises beauty processing on the second image; the step of controlling the smart mirror to generate interaction with the current user and output interactive information according to the input of the current user comprises:
processing the second image to obtain a beauty face area of the current user; processing the beauty face area to obtain beauty feature points of the beauty face area; and performing the beauty processing the second image according to the input of the current user and the beauty feature points to obtain a beauty image.

6 . The control method according to claim 5 , wherein the beauty processing comprises one or more of whitening filter, ruddy filter, thin face module and big eyes module.

7 . The control method according to claim 3 , wherein the interaction comprises virtual trial makeup processing on the second image, the smart mirror comprises a makeup material library, and the step of controlling the smart mirror to generate interaction with the current user and output interactive information according to the input of the current user comprises:
processing the second image to obtain a trial makeup face area of the current user; processing the test-cosmetic face area to obtain trial makeup feature points of the trial makeup face area; determining makeup material according to the input of the current user; and matching and fusing the makeup material with the second image according to the trial makeup feature points, to obtain a virtual trial makeup image.

8 . The control method according to claim 7 , wherein the makeup material library comprises one or more of eye shadow material, eyeliner material, blusher material, lip gloss material and eyebrow material.

9 . The control method according to claim 3 , wherein the interaction comprises 2D mask rendering on the second image, and the smart mirror comprises a 2D mask material library; the step of controlling the smart mirror to generate interaction with the current user and output interactive information according to the input of the current user comprises:
processing the second image to obtain a 2D mask rendering face area of the current user; processing the 2D mask rendering face area to obtain 2D mask rendering feature points of the 2D mask rendering face area; determining 2D mask material according to the input of the current user; and matching and fusing the 2D mask material with the second image according to the 2D mask rendering feature points to obtain a 2D mask rendering image.

10 . The control method according to claim 3 , wherein the interaction comprises 3D cartoon image rendering on the second image, and the smart mirror comprises a 3D engine, a universal 3D face model and a 3D cartoon image material library; the step of controlling the smart mirror to generate interaction with the current user and output interactive information according to the input of the current user comprises:
processing the second image to obtain a 3D cartoon image rendering face area of the current user; processing the 3D cartoon image rendering face area to obtain 3D cartoon image rendering feature points of the 3D cartoon image rendering face area; obtaining a first pose parameter of the current user according to the universal 3D face model and the 3D cartoon image rendering feature points; determining 3D cartoon image material according to the input of the current user; and controlling the 3D engine to perform 3D cartoon image rendering on the second image according to the first pose parameter and the 3D cartoon image material.

11 . The control method according to claim 3 , wherein the interaction comprises virtual glasses rendering on the second image, and the smart mirror comprises a 3D engine, a universal 3D face model and a virtual glasses material library; the step of controlling the smart mirror to generate interaction with the current user and output interactive information according to the input of the current user comprises:
processing the second image to obtain a virtual glasses rendering face area of the current user; processing the virtual glasses rendering face area to obtain virtual glasses rendering feature points of the virtual glasses rendering face area; obtaining a second pose parameter of the current user according to the universal 3D face model and the virtual glasses rendering feature points; determining virtual glasses material according to the input of the current user; and controlling the 3D engine to perform virtual glasses rendering on the second image according to the second pose parameter and the virtual glasses material.

12 . The control method according to claim 3 , wherein the interaction comprises virtual hairstyle rendering on the second image, and the smart mirror comprises a 3D engine, a universal 3D face model and a virtual hairstyle material library; the step of controlling the smart mirror to generate interaction with the current user and output interactive information according to the input of the current user comprises:
processing the second image to obtain a virtual hairstyle rendering face area of the current user; processing the virtual hairstyle rendering face area to obtain virtual hairstyle rendering feature points of the virtual hairstyle rend

Assignees

• Midea Group Co Ltd

Inventors

• Yu Dahai
• Quan Yongbing
• Li Jianping
• Zhou Junyang
• Song Jianfeng

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2200/04

  involving 3D image data · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A47G1/02

  Mirrors used as equipment (hand, pocket, or shaving mirrors A45D42/00; {optical viewing means for vehicles B60R1/00}) · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title