Image device for determining an invalid depth information of a depth image and operation method thereof

Patent metadata

Field	Value
Publication number	US-10096170-B2
Application number	US-201414297592-A
Country	US
Kind code	B2
Filing date	Jun 5, 2014
Priority date	Jun 6, 2013
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

An image device for determining an invalid depth information of a depth image includes a first sensor, a second sensor, a pre-processing module, a depth map engine, and an error determination unit. The first sensor captures first images and the second sensor captures second images. The pre-processing module generates a reference image according to each first image of the first images and a target image corresponding to the reference image according to a second image of the second images. The depth map engine generates a first depth image according to the reference image and the target image. The error determination unit sets gray level values of pixels of invalid areas of the first depth image to be a predetermined value to generate a second depth image, wherein pixels of the second depth image with the predetermined value have the invalid depth information.

First claim

What is claimed is:

1. An image device for determining an invalid depth information of a depth image, the image device comprising:
a first sensor capturing first images;
a second sensor capturing second images;
a pre-processor coupled to the first sensor and the second sensor, wherein the pre-processor generates a reference image according to each first image of the first images and a target image corresponding to the reference image according to a second image of the second images, defines invalid areas of the reference image according to an image range of the each first image, the invalid areas of the reference image are located outside the image range of the each first image, sets a first predetermined gray level value to pixels of the invalid areas of the reference image, and the second image corresponds to the each first image;
a depth map generation circuit coupled to the pre-processor, wherein the depth map generation circuit generates a first depth image according to the reference image and the target image; and
an error determination circuit coupled to the pre-processor and the depth map generation circuit, wherein the error determination circuit defines invalid areas of the first depth image according to the invalid areas of the reference image, and sets a second predetermined gray level value to pixels of the invalid areas of the first depth image to define invalid areas of a second depth image, wherein the invalid areas of the second depth image are neglected.

2. The image device of claim 1 , wherein the pre-processor comprises:
a synchronization circuit coupled to the first sensor and the second sensor, wherein the synchronization circuit simultaneously outputs the each first image and the second image;
an image processor coupled to the synchronization circuit, wherein the image processor executes an image processing on the each first image and the second image respectively to generate a processed first image and a processed second image; and
a rectifier coupled to the image processor, wherein the rectifier executes a rectification processing on the processed first image and the processed second image respectively to generate the reference image and the target image, and sets the first predetermined gray level value to the pixels of the invalid areas of the reference image.

3. The image device of claim 2 , wherein the image processing is a combination of color space conversion, adjusting luminance, adjusting resolution, noise cancellation, edge enhancement, interpolation and adjusting contrast.

4. The image device of claim 2 , wherein the rectification processing is a coordinate transforming processing, and the reference image and the target image meet an epipolar constrain.

5. The image device of claim 1 , wherein the first images are left eye images and the second images are right eye images.

6. The image device of claim 1 , wherein the first images and the second images are color images or gray level images.

7. The image device of claim 1 , wherein when gray level values of pixels of the reference image inside the image range of the each first image are equal to the first predetermined gray level value, the gray level values of the pixels of the reference image inside the image range of the each first image are set to be a third predetermined gray level value, wherein the first predetermined gray level value is different from the third predetermined gray level value.

8. An operation method for determining an invalid depth information of a depth image, wherein the operation method is applied to an image device, the image device comprises a first sensor, a second sensor, a pre-processor, a depth map generation circuit, and an error determination circuit, and the pre-processor comprises a synchronization circuit, an image processor, and a rectifier, the operation method comprising:
the first sensor capturing first images and the second sensor capturing second images;
the pre-processor generating a reference image according to each first image of the first images and a target image corresponding to the reference image according to a second image of the second images;
the pre-processor defining invalid areas of the reference image according to an image range of the each first image, wherein the invalid areas of the reference image are located outside the image range of the each first image;
the depth map generation circuit generating a first depth image according to the reference image and the target image;
the error determination circuit defining invalid areas of the first depth image according to the invalid areas of the reference image; and
the error determination circuit setting a second predetermined gray level value to pixels of the invalid areas of the first depth image to define invalid areas of a second depth image, wherein the invalid areas of the second depth image are neglected.

9. The operation method of claim 8 , wherein the pre-processor generating the reference image and the target image, and determining the invalid areas of the reference image comprises:
the synchronization circuit simultaneously outputting the each first image and the second image;
the image processor executing an image processing on the each first image and the second image respectively to generate a processed first image and a processed second image; and
the rectifier executing a rectification processing on the processed first image and the processed second image respectively to generate the reference image and the target image, and setting the first predetermined gray level value to the pixels of the invalid areas of the reference image.

10. The operation method of claim 9 , wherein the image processing comprises a combination of color space conversion, adjusting luminance, adjusting resolution, noise cancellation, edge enhancement, interpolation and adjusting contrast.

11. The operation method of claim 9 , wherein the rectification processing is a coordinate transforming processing, and the reference image and the target image meet an epipolar constrain.

12. The operation method of claim 8 , wherein the first images are left eye images and the second images are right eye images.

13. The operation method of claim 8 , wherein the first images and the second images are color images or gray level images.

14. The operation method of claim 8 , wherein when gray level values of pixels of the reference image inside the image range of the each first image are equal to the first predetermined gray level value, the gray level values of the pixels of the reference image inside the image range of the each first image are set to be a third predetermined gray level value, wherein the first predetermined gray level value is different from the third predetermined gray level value.

Assignees

• Etron Tech Inc
• Eys3D Microelectronics Co

Inventors

• Lee Chi-Feng

Classifications

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N2213/003

  Aspects relating to the "2D+depth" image format · CPC title

• G06T7/593 Primary

  from stereo images · CPC title

• H04N13/00

  Stereoscopic video systems; Multi-view video systems; Details thereof · CPC title