Depth image generation utilizing depth information reconstructed from an amplitude image

Patent metadata

Field	Value
Publication number	US-2016247286-A1
Application number	US-201414378119-A
Country	US
Kind code	A1
Filing date	Aug 11, 2014
Priority date	Feb 7, 2014
Publication date	Aug 25, 2016
Grant date	—

Abstract

An image processing system comprises an image processor having image processing circuitry and an associated memory. The image processor is configured to identify a region of interest in an amplitude image, to detect one or more relatively low gradient regions in the region of interest, to reconstruct depth information for said one or more relatively low gradient regions, to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest, and to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information. The image processor in some embodiments is adapted for coupling to an active lighting image sensor, such as an infrared sensor that does not provide depth information corresponding to the amplitude image, or an SL or ToF sensor that provides depth information corresponding to the amplitude image.

First claim

What is claimed is:

1 . A method comprising steps of:
identifying a region of interest in an amplitude image; detecting one or more relatively low gradient regions in the region of interest; reconstructing depth information for said one or more relatively low gradient regions in the region of interest; extending the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and generating a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information; wherein the steps are implemented in an image processor comprising a processor coupled to a memory.

2 . The method of claim 1 wherein said one or more relatively low gradient regions comprise regions having respective gradients at or below a specified gradient threshold.

3 . The method of claim 1 wherein said one or more relatively low gradient regions comprise regions having respective substantially zero gradients.

4 . The method of claim 1 wherein identifying a region of interest comprises generating a binary region of interest mask in which pixels within the region of interest all have a first binary value and pixels outside the region of interest all have a second binary value complementary to the first binary value.

5 . The method of claim 1 wherein reconstructing depth information for said one or more relatively low gradient regions comprises reconstructing depth information for said one or more relatively low gradient regions but not for other portions of the region of interest.

6 . The method of claim 1 wherein the amplitude image comprises a luminance image generated by an active lighting imager.

7 . The method of claim 6 wherein the active lighting imager comprises one of an active lighting infrared image sensor, an SL image sensor and a ToF image sensor.

8 . The method of claim 1 wherein reconstructing depth information for said one or more relatively low gradient regions in the region of interest comprises:
determining at least one coefficient that relates amplitude to depth for a particular image sensor configuration; and
utilizing the coefficient to estimate one or more depth values from one or more corresponding amplitude values for respective pixels of said one or more relatively low gradient regions.

9 . The method of claim 8 wherein the determined coefficient comprises an element of a coefficient function that relates amplitude to depth for a given pixel as a function of a distance between that pixel and an approximate center of an image frame in the particular image sensor configuration.

10 . The method of claim 1 wherein extending the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest comprises computing one or more depth values for pixels outside of said one or more relatively low gradient regions as a function of depth values determined for pixels within said one or more relatively low gradient regions.

11 . The method of claim 10 wherein a depth value for a given pixel outside of said one or more relatively low gradient regions is computed utilizing a mean of a plurality of depth values for respective pixels within said one or more relatively low gradient regions.

12 . The method of claim 1 wherein identifying a region of interest comprises identifying the region of interest utilizing the amplitude image and a corresponding coarse depth image.

13 . The method of claim 1 wherein generating a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information further comprises:
generating a reconstructed depth image comprising at least portions of the reconstructed depth information and the extended reconstructed depth information; and
combining a coarse depth image with the reconstructed depth image.

14 . An article of manufacture comprising a computer-readable storage medium having computer program code embodied therein, wherein the computer program code when executed in the image processor causes the image processor to perform the method of claim 1 .

15 . An apparatus comprising:
an image processor adapted for coupling to an image sensor; the image processor comprising image processing circuitry and an associated memory; wherein the image processor is configured: to identify a region of interest in an amplitude image; to detect one or more relatively low gradient regions in the region of interest; to reconstruct depth information for said one or more relatively low gradient regions in the region of interest; to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information.

16 . An integrated circuit comprising the apparatus of claim 15 .

17 . An imager comprising:
an image sensor; and an image processor coupled to the image sensor; wherein the image processor is configured: to identify a region of interest in an amplitude image; to detect one or more relatively low gradient regions in the region of interest; to reconstruct depth information for said one or more relatively low gradient regions in the region of interest; to extend the reconstructed depth information beyond said one or more relatively low gradient regions to additional pixels of the region of interest; and to generate a depth image utilizing at least portions of the reconstructed depth information and the extended reconstructed depth information.

18 . The imager of claim 17 wherein the image sensor comprises an active lighting infrared image sensor that does not provide depth information corresponding to the amplitude image.

19 . The imager of claim 17 wherein the image sensor comprises an SL or ToF image sensor that provides depth information corresponding to the amplitude image.

20 . An image processing system comprising the imager of claim 17 .

Assignees

• Lsi Corp

Inventors

• Mazurenko Ivan L
• Radovanovic Nikola
• Parkhomenko Denis V
• Kholodenko Alexander B
• Parfenov Denis V

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T7/0057 Primary

  Physics · mapped topic

• G06T7/0081

  Physics · mapped topic

• G06K9/00335

  Physics · mapped topic

• G06T7/73

  using feature-based methods · CPC title