Using multiple cameras to stitch a consolidated 3D depth map

Patent metadata

Field	Value
Publication number	US-10178370-B2
Application number	US-201615383683-A
Country	US
Kind code	B2
Filing date	Dec 19, 2016
Priority date	Dec 19, 2016
Publication date	Jan 8, 2019
Grant date	Jan 8, 2019

Abstract

Each of plural devices includes a laser emitter and a camera for detecting reflections of laser light emitted by the device, so that plural of the devices can generate their own depth maps showing images within the field of view of their cameras. The resulting 3D depth maps from multiple cameras at potentially arbitrary locations can be aggregated to create a more accurate 3D depth map of the area covered by all the individual cameras. Each device/camera may be assigned a unique identification such as a unique number and a mechanism to identify, either electronically or visually, devices in its field of view. Using this information, the relative locations of the cameras can be calculated for purposes of aggregating the multiple depth maps.

First claim

What is claimed is:

1. A device comprising:
at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor to:
generate, using a first device, at least a first image of second and third devices in a field of view of a camera;
the first image being a first 3D depth map;
receive from at least one of the second and third devices information useful for generating a second 3D depth map including an image of the first device; and
generate an aggregate 3D depth map using the first and second 3D depth maps.

2. The device of claim 1 , wherein the instructions are executable to:
exchange device identifying information with the second and third devices.

3. The device of claim 1 , wherein the information useful for generating a second 3D depth map includes the device identifying information, and the first and second 3D depth maps are generated by a single camera located in respective first location and second locations.

4. The device of claim 1 , wherein the information useful for generating a second 3D depth map includes an image of the first device taken by the second device.

5. The device of claim 1 , wherein the instructions are executable to:
receive from at least one of the second and third devices information useful for generating a third 3D depth map including an image of the first device; and
generate an aggregate 3D depth map using the first, second, and third 3D depth maps.

6. The device of claim 1 , wherein the instructions are executable to:
determine a distance to a device corresponding to an image in the first 3D depth map at least in part based on a size of the image.

7. The device of claim 1 , wherein the instructions are executable to:
alter aspects of images in the first and second 3D depth maps for a reference location to render altered images in the aggregate 3D depth map.

8. An assembly comprising:
plural computerized devices;
each computerized device comprising at least one laser emitter configured to output signals useful for generating at least one three dimensional (3D) depth map;
a first one of the computerized devices being programmed with instructions to:
generate, using the first device, at least a first image of second and third devices in a field of view of a camera;
the first image being a first 3D depth map;
receive from at least one of the second and third devices information useful for generating a second 3D depth map including an image of the first device; and
generate an aggregate 3D depth map using the first and second 3D depth maps.

9. The assembly of claim 8 , wherein the instructions are executable to:
exchange device identifying information with the second and third devices.

10. The assembly of claim 8 , wherein the information useful for generating a second 3D depth snap includes the device identifying information, and the first and second 3D depth maps are generated by a single camera located in respective first location and second locations.

11. The assembly of claim 8 , wherein the information useful for generating a second 3D depth map includes an image of the first device taken by the second device.

12. The assembly of claim 8 , wherein the instructions are executable to:
receive from at least one of the second and third devices information useful for generating a third 3D depth map including an image of the first device; and
generate an aggregate 3D depth map using the first, second, and third 3D depth maps.

13. The assembly of claim 8 , wherein the instructions are executable to:
determine a distance to a device corresponding to an image in the first 3D depth map at least in part based on a size of the image.

14. The assembly of claim 8 , wherein the instructions are executable to:
alter aspects of images in the first and second 3D depth maps for a reference location to render altered images in the aggregate 3D depth map.

15. A method comprising:
generating, using a first device, at least a first image of second and third devices in a field of view of a camera;
the first image being a first 3D depth map;
receiving from at least one of the second and third devices information useful for generating a second 3D depth map including an image of the first device; and
generating an aggregate 3D depth map using the first and second 3D depth maps.

16. The method of claim 15 , comprising:
exchanging device identifying information with the second and third devices.

17. The method of claim 15 , wherein the information useful for generating a second 3D depth map includes the device identifying information, and the first and second 3D depth maps are generated by a single camera located in respective first location and second locations.

18. The method of claim 15 , wherein the information useful for generating a second 3D depth map includes an image of the first device taken by the second device.

19. The method of claim 15 , comprising:
receiving from at least one of the second and third devices information useful for generating a third 3D depth map including an image of the first device; and
generating an aggregate 3D depth map using the first, second, and third 3D depth maps.

20. The method of claim 15 , comprising:
altering aspects of images in the first and second 3D depth maps for a reference location to render altered images in the aggregate 3D depth map.

Assignees

• Sony Corp
• Sony Interactive Entertainment Inc

Inventors

• Shintani Peter
• Gonzalez Hernandez Jose Omar
• Clay William
• Espinosa Pablo Antonio
• Ansfield Fred
• Mohapatra Bibhudendu
• Resch Keith
• Usami Morio

Classifications

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• H04N13/271

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• H04N13/156 Primary

  Mixing image signals · CPC title

• G01B11/00

  Measuring arrangements characterised by the use of optical techniques · CPC title

• H04N5/33

  Transforming infrared radiation (cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20) · CPC title