Method and apparatus for reconstructing 3D face with stereo camera

Patent metadata

Field	Value
Publication number	US-10043278-B2
Application number	US-201514609887-A
Country	US
Kind code	B2
Filing date	Jan 30, 2015
Priority date	Feb 10, 2014
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

Provided is a method and apparatus for reconstructing a three-dimensional (3D) face based on a stereo camera, the method including: acquiring n images of a target by controlling a plurality of stereo cameras in response to an image acquirement request, wherein n denotes a natural number; extracting n face regions from the n images, respectively; and reconstructing a viewpoint-based face image based on the n face regions.

First claim

What is claimed is:

1. A method of reconstructing a three-dimensional (3D) face from images captured by a plurality of stereo cameras, the method comprising:
simultaneously acquiring n images of a target by controlling the plurality of stereo cameras in response to an image acquirement request, wherein n denotes a natural number equal to or greater than 4;
extracting n face regions from the n images, respectively;
reconstructing a viewpoint-based face image based on the n face regions;
determining an estimated error point in the viewpoint-based face image based on a 3D coordinate value or a disparity value of a point in the viewpoint-based face image that exceed respective predetermined error ranges;
removing the estimated error point from the viewpoint-based face image; and
performing matching processing on the viewpoint-based face image from which the estimated error point is removed,
wherein each stereo camera comprises a plurality of digital single-lens reflex (DSLR) cameras.

2. The method of claim 1 , wherein the reconstructing comprises:
generating a disparity map about a stereo camera used to acquire an image; and
generating a 3D coordinate value at which a face region is positioned on the viewpoint-based face image, using the disparity map.

3. The method of claim 1 , wherein the removing of the estimated error point comprises:
calculating a correspondence point of a predetermined point within a first face image and the disparity value of the correspondence point, from a second face image having a viewpoint different from a viewpoint of the first face image;
comparing the disparity value of the correspondence point with a disparity value of a neighboring correspondence point identified from the second face image; and
removing the correspondence point from the second face image when a difference between the disparity value of the correspondence point and the disparity value of the neighboring correspondence point exceeds the predetermined error range.

4. The method of claim 1 , wherein the removing of the estimated error point comprises:
identifying a point having the disparity value or the 3D coordinate value exceeding the respective predetermined error ranges from the viewpoint-based face image, using face shape information of the target; and
removing the identified point from the viewpoint-based face image.

5. The method of claim 1 , wherein the performing of the matching processing comprises:
merging the viewpoint-based face images into a single face image;
generating a texture map about the merged face image using visual field portions and a non-visual field portions of the plurality of stereo cameras; and
processing the entire tone of the generated texture map to be matched.

6. The method of claim 5 , wherein the merging comprises merging an overlapping region between the viewpoint-abased face images to be merged, using an average value.

7. The method of claim 1 , wherein the acquiring comprises:
controlling the plurality of stereo cameras to simultaneously capture an image of the target by generating a synchronization signal for each of the plurality of stereo cameras; and
storing and maintaining, in a memory, each captured image in association with identification information.

8. The method of claim 1 , wherein the image acquirement request is generated through a hardware method using wired/wireless equipment or a software method using a button click on a personal computer (PC) screen.

9. The method of claim 1 , wherein the extracting comprises:
modeling a background region included in an image, based on a guideline for each stereo camera; and
extracting a face region by removing a first background portion from a region acquired through modeling.

10. The method of claim 9 , wherein the extracting of the face region further comprises:
further removing a second background portion included in a remaining region from which the first background portion is removed, using face shape information of the target; or
extracting the face region by adding, to the remaining region, a face portion included in the removed first background portion.

11. The method of claim 1 , further comprising:
generating a mask image of the extracted face region.

12. An apparatus for reconstructing a three-dimensional (3D) face from images captured by a plurality of stereo cameras, the apparatus comprising a non-transitory computer-readable medium which, when executed by the computer, performs the following method:
simultaneously acquiring n images of a target by controlling the plurality of stereo cameras in response to an image acquirement request, wherein n denotes a natural number equal to or greater than 4;
extracting n face regions from the n images, respectively;
reconstructing a viewpoint-based face image based on the n face regions;
determining an estimated error point in the viewpoint-based face image based on a 3D coordinate value or a disparity value of a point in the viewpoint-based face image that exceed respective predetermined error ranges;
removing the estimated error point from the viewpoint-based face image; and
performing matching processing on the viewpoint-based face image from which the estimated error point is removed,
wherein each stereo camera comprises a plurality of digital single-lens reflex (DSLR) cameras.

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Kim Kap Kee
• Yoon Seung Uk
• Hwang Bon Woo
• Lim Seong Jae
• Jun Hye Ryeong
• Choi Jin Sung
• Koo Bon Ki

Classifications

• H04N13/282

  for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• G06T7/593 Primary

  from stereo images · CPC title

• H04N13/117

  the virtual viewpoint locations being selected by the viewers or determined by viewer tracking · CPC title

• G06T2207/30201

  Face · CPC title