Generating 3D depth map using parallax

Patent metadata

Field	Value
Publication number	US-10979695-B2
Application number	US-201916653985-A
Country	US
Kind code	B2
Filing date	Oct 15, 2019
Priority date	Oct 31, 2017
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

Parallax views of objects are used to generate 3D depth maps of the objects using time of flight (TOF) information. In this way, the deleterious effects of multipath interference can be reduced to improve 3D depth map accuracy without using computationally intensive algorithms.

First claim

What is claimed is:

1. A device comprising:
at least one processor configured with instructions to:
receive at least a first camera image of at least one object from a first image generation location;
receive at least a camera image of the at least one object from a second image generation location, the first and second image generation locations rendering parallax views of the object;
comparing the first camera image to the second camera image to identify at least one discrepancy;
use at least one of the first and second camera images to render a three-dimensional (3D) depth map; and
responsive to identifying a discrepancy, remove at least one element of the 3D depth map, the discrepancy being identified at least in part by one or more of: comparing relative sizes of respective objects in the respective first and second camera images, identifying that at least one object presents a size that exceeds a threshold size, identifying that a flat surface with a size larger than a threshold size is imaged.

2. The device of claim 1 , wherein the instructions are executable to identify the discrepancy at least in part by comparing relative sizes of respective objects in the respective first and second camera images.

3. The device of claim 1 , wherein the instructions are executable to identify the discrepancy at least in part by identifying that at least one object presents a size that exceeds a threshold size.

4. The device of claim 1 , wherein the instructions are executable to establish the second image generation location by translationally moving an imaging object supporting a camera generating the images.

5. The device of claim 1 , wherein the instructions are executable to:
establish the second image generation location by moving a camera relative to an imaging object supporting the camera.

6. The device of claim 1 , wherein the instructions are executable to:
use the first and second camera images to render a three-dimensional (3D) depth map by consolidating the first and second camera images.

7. The device of claim 1 , wherein the instructions are executable to:
use the first and second camera images to render a three-dimensional (3D) depth map by determining which of the first and second camera images exhibits less noise, and using the image exhibiting less noise to establish the 3D depth map.

8. The device of claim 1 , wherein the instructions are executable to identify the discrepancy at least in part by identifying that a flat surface with a size larger than a threshold size is imaged.

9. A device comprising:
at least one processor configured with instructions to:
receive at least a first camera image of at least one object from a first image generation location;
receive at least a camera image of the at least one object from a second image generation location; and
use the first and second camera images to render a three-dimensional (3D) depth map, wherein the instructions are executable to identify whether at least one discrepancy exists between the first and second camera images exceeds a threshold at least in part by executing object recognition on at least a first object in at least one of the camera images, wherein responsive to the discrepancy meeting the threshold, the device implicitly assumes that a light path that created the discrepancy is due to a wall or ceiling reflection and in response removes excess mapping points from the 3D depth map that produced the discrepancy.

10. The device of claim 9 , wherein the representations comprise a two-dimensional (2D) images of the object.

11. The device of claim 9 , wherein the representations are a three-dimensional (3D) depth map of the object.

12. The device of claim 9 , wherein the first image comprises a 2D camera image and the instructions are executable to:
compare the first image to the 3D depth map to identify a discrepancy; and
responsive to identifying a discrepancy, remove at least one element of the 3D depth map.

13. The device of claim 9 , wherein the instructions are executable to establish the second image generation location by translationally moving an imaging object supporting a camera generating the images.

14. The device of claim 9 , wherein the instructions are executable to:
use the first and second camera images to render a three-dimensional (3D) depth map by consolidating the first and second camera images.

15. The device of claim 9 , wherein the instructions are executable to:
use the first and second camera images to render a three-dimensional (3D) depth map by determining which of the first and second camera images exhibits less noise, and using the image exhibiting less noise to establish the 3D depth map.

16. A device comprising:
at least one processor configured with instructions to:
receive at least a first camera image of at least one object from a first image generation location;
receive at least a camera image of the at least one object from a second image generation location;
use the first and second camera images to render a three-dimensional (3D) depth map, wherein the instructions are executable to:
establish the second image generation location by translationally moving an imaging object supporting the first camera generating the images away from an interfering flat surface and turning the first camera toward an object.

17. The device of claim 16 , wherein the first image comprises a 2D camera image and the instructions are executable to:
compare the first image to the 3D depth map to identify a discrepancy; and
responsive to identifying a discrepancy, remove at least one element of the 3D depth map.

18. The device of claim 16 , wherein the instructions are executable to determine which of the first and second camera images exhibits less noise, and use the image exhibiting less noise to establish the 3D depth map.

19. The device of claim 16 , wherein the instructions are executable to:
use the first and second camera images to render a three-dimensional (3D) depth map by consolidating the first and second camera images.

20. The device of claim 16 , wherein the second image generation location is established by moving a camera relative to an imaging object supporting the camera.

Assignees

• Sony Corp
• Sony Interactive Entertainment Inc

Inventors

• Shintani Peter
• Usami Morio
• Shikama Kazuyuki
• Resch Keith

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/593

  from stereo images · CPC title

• H04N13/271 Primary

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• G01S17/08

  for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48) · CPC title