Display apparatus and control method for adjusting the eyes of a photographed user

Patent metadata

Field	Value
Publication number	US-9536133-B2
Application number	US-201414150265-A
Country	US
Kind code	B2
Filing date	Jan 8, 2014
Priority date	Jan 9, 2013
Publication date	Jan 3, 2017
Grant date	Jan 3, 2017

Abstract

A display apparatus is provided. The display apparatus includes a photographing unit configured to photograph the shape of a face; a detector is configured to detect a direction and angle of the face shape; a transformer is configured to mix the photographed face shape and a 3D face model and to transform the mixed face shape by using the detected direction and angle of the face shape; and an output interface is configured to output the transformed face shape.

First claim

What is claimed is:

1. A display apparatus comprising:
a photographing unit configured to photograph a shape of a user face;
a detector configured to detect a direction and angle of the face shape;
a transformer configured to mix the shape of the photographed face and a 3D face model and transform the mixed face shape by using the detected direction and angle of the face shape; and
an output interface configured to output an image of the transformed face shape,
wherein the transformer transforms the mixed face shape by adjusting a gaze direction in the image so that the gaze direction faces forward.

2. The display apparatus of claim 1 , wherein
the transformer is configured to additionally transform a corresponding face boundary portion of the transformed face shape by using information related to a face boundary portion of the photographed face shape.

3. The display apparatus of claim 2 , wherein
the transformer is configured to detect a first boundary which is formed by a predetermined gap inward from the face shape and a second boundary which is formed by a predetermined gap outward from the face shape, based on the photographed face shape and the face boundary of the transformed face shape, and
additionally transforms a corresponding portion between the first boundary and the second boundary of the transformed face shape by using information related to a portion between the first boundary and the second boundary of the shape of the photographed face.

4. The display apparatus of claim 1 , wherein
the detector is configured to extract feature points from the photographed face shape,
generate a reference 3D face model in the form of a mesh by extracting feature points from a stored reference 3D face model, and
generate a 3D face model in the form of a mesh which corresponds to the photographed face shape by projecting the reference 3D face model in the form of a mesh and matching the corresponding feature points.

5. The display apparatus of claim 4 , wherein
the detector is configured to detect the direction and angle of the face shape by using a difference in position of predetermined portions of the reference 3D face model and the generated 3D face model.

6. The display apparatus of claim 4 , wherein
the transformer is configured to project the predetermined portion of the 3D face model in the form of a mesh, and then mixes the corresponding unit meshes with unit meshes of the photographed face shape in order to generate a mixed face shape in the form of a mesh.

7. The display apparatus of claim 6 , wherein
the predetermined portion comprises an eye portion.

8. The display apparatus of claim 1 , further comprising:
a display configured to display the transformed face shape.

9. The display apparatus of claim 1 , further comprising:
a communicator configured to perform at least one of a transmitting function of the transformed face shape data and a receiving function of face shape data of which transformation is performed in an external display apparatus.

10. A method of controlling a display apparatus, the method comprising:
photographing a shape of a face;
detecting a direction and angle of the face shape;
mixing the photographed face shape and a 3D face model;
transforming the mixed face shape by using the detected direction and angle of the face shape; and
outputting an image comprising the shape of the transformed face,
wherein the transforming the mixed face shape comprises transforming the mixed face shape by adjusting a gaze direction in the image so that the gaze direction faces forward.

11. The control method of claim 10 , wherein
the transforming the mixed face shape comprises additionally transforming a corresponding face boundary portion of the transformed face shape by using information related to a face boundary portion of the photographed face shape.

12. The display method of claim 11 , wherein
the transforming the mixed face shape comprises detecting a first boundary which is formed by a predetermined gap inward from the face shape and a second boundary which is formed by a predetermined gap outward from the face shape, based on the photographed face shape and the face boundary of the transformed face shape, and
additionally transforming a corresponding portion between the first boundary and the second boundary of the transformed face shape by using information related to a portion between the first boundary and the second boundary of the shape of the photographed face.

13. The control method of claim 10 , wherein
the detecting direction and angle of the face shape comprises extracting feature points from the shape of the photographed face,
generating a reference 3D face model in the form of a mesh by extracting feature points from a stored reference 3D face model, and
generating a 3D face model in the form of a mesh which corresponds to the photographed face shape by projecting the reference 3D face model in the form of the mesh and matching corresponding feature points.

14. The display method of claim 13 , wherein
the detecting direction and angle of the face shape comprises detecting the direction and angle of the face shape by using a difference in position of predetermined portions of the reference 3D face model and the generated 3D face model.

15. The display method of claim 13 , wherein
the transforming the mixed face shape comprises projecting the predetermined portion of the 3D face model in the form of a mesh, and then mixes corresponding unit meshes with unit meshes of the photographed face shape in order to generate a mixed face shape in the form of a mesh.

16. The display method of claim 15 , wherein
the predetermined portion comprises an eye portion.

17. The display method of claim 10 , further comprising:
displaying the transformed face shape.

18. The display method of claim 10 , further comprising:
performing at least one of a transmitting function of the transformed face shape data and a receiving function of face shape data of which transformation is performed in an external display apparatus.

19. A display apparatus comprising:
a detector configured to detect a direction and angle of a face shape;
a transformer configured to mix the shape of the face and a reference 3D face model and transform the mixed face shape using the detected direction and angle of the face shape; and
an output interface configured to output an image of the transformed face shape,
wherein the transformer transforms the mixed face shape by adjusting a gaze direction in the image so that the gaze direction faces forward.

20. The display apparatus of claim 19 , further comprising a photographing unit configured to photograph the shape of a user face.

21. The display apparatus of claim 20 , wherein the transformer is configured to additionally transform a corresponding face boundary portion of the transformed face shape by using information related to a face boundary portion of the photographed face shape.

22. The display apparatus of claim 20 , wherein the detector is configured to extract feature points from the shape of the photographed face.

23. The display apparatus of claim 22 , wherein the detector is configured to generate a reference 3D face model in the form of a mesh by extracting feature points from a stored reference 3D face model.

24. The display apparatus of claim 23 , wherein the detector is configured to generate a 3D face model in the form of a mes

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Do-Wan
• Kim Sung-Yeol
• Kim Sung-Jin
• Kim Ju-Hee
• Lee Jin-Sung

Classifications

• G06K9/00268 Primary

  Physics · mapped topic

• G06K9/00208

  Physics · mapped topic

• G06T19/00 Primary

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title

• G06V40/168 Primary

  Feature extraction; Face representation · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title