Depth assisted scene recognition for a camera

Patent metadata

Field	Value
Publication number	US-2016277724-A1
Application number	US-201414433174-A
Country	US
Kind code	A1
Filing date	Apr 17, 2014
Priority date	Apr 17, 2014
Publication date	Sep 22, 2016
Grant date	—

Abstract

A digital camera is operated by calculating a digital depth map of a scene that is received by the camera. Based on the digital depth map of the scene, one of the available scene mode settings is automatically selected. Related methods, devices, and/or computer program product are described.

First claim

1 . A method of operating a camera comprising:
calculating a digital depth map of a scene that is received by the camera; and automatically selecting one of a plurality of scene mode settings for the scene based on the digital depth map of the scene.

2 . The method of claim 1 ,
wherein automatically selecting one of the plurality of scene mode settings is preceded by determining an initial scene mode setting out of the plurality of scene mode settings, based on non-depth information related to the scene; and wherein automatically selecting one of the plurality of scene mode settings comprises automatically changing the initial scene mode setting based on the digital depth map of the scene.

3 . The method of claim 1 ,
wherein automatically selecting one of the plurality of scene mode settings comprises automatically selecting one of a plurality of scene mode settings for the scene based on the digital depth map of the scene and further based on non-depth information.

4 . The method of claim 1 ,
wherein the scene comprises a plurality of pixels, and wherein the calculating the digital depth map comprises calculating a depth value for one or more of the plurality of pixels in the scene.

5 . The method of claim 1 ,
wherein the camera comprises a plurality of independent image capturing systems; and wherein the calculating the digital depth map comprises calculating a digital depth map of a scene from at least two of the plurality of independent image capturing systems.

6 . The method of claim 5 , wherein the calculating is performed using only two of the plurality of the independent image capturing systems.

7 . The method of claim 1 ,
wherein the calculating the digital depth map comprises calculating a plurality of digital depth maps, a respective one of which is related to a respective frame of the scene; and wherein automatically selecting one of the plurality of scene mode settings comprises automatically selecting one of the plurality of scene mode settings based on the plurality of digital depth maps.

8 . The method of claim 1 ,
wherein the calculating the digital depth map comprises calculating a plurality of digital depth maps, a respective one of which is related to a respective frame of the scene; wherein automatically selecting one of the plurality of scene mode settings comprises comparing two or more of the plurality of digital depth maps to determine the presence of motion in the scene; and wherein automatically selecting one of the plurality of scene mode settings further comprises automatically selecting the scene mode setting based on at least one of the plurality of digital depth maps and the presence of motion in the scene.

9 . The method of claim 1 , wherein automatically selecting one of the plurality of scene mode settings for the scene based on the digital depth map of the scene comprises:
identifying one or more objects in the scene based on the digital depth map; assigning depth values to each of the one or more of the objects in the scene; weighting the one or more objects in the scene based on the assigned depth values; and automatically selecting the scene mode setting based on the respective weighting of the one or more objects.

10 . The method of claim 9 , wherein identifying the one or more objects in the scene based on the digital depth map comprises:
classifying one or more of a plurality of pixels in the scene into depth ranges based on the digital depth map; and identifying the one or more objects in the scene based on the classification of the one or more of the plurality of pixels in the scene into depth ranges.

11 . The method of claim 9 , wherein weighting the one or more objects in the scene based on the assigned depth values comprises:
determining the respective type of respective ones of the one or more objects in the scene; assigning priorities to the one or more objects in the scene that were identified based on the determined respective type of the respective ones of the one or more objects in the scene; and weighting the one or more objects in the scene based on the priorities that were assigned to the one or more objects.

12 . The method of claim 1 , wherein automatically selecting one of the plurality of scene mode settings for the scene based on the digital depth map of the scene comprises:
classifying one or more of the plurality of pixels in the scene into depth ranges based on the digital depth map; weighting one or more pixels in the scene based on the classification of the one or more of the plurality of pixels into depth ranges; and automatically selecting the scene mode setting based on the weighting of the one or more pixels.

13 . The method of claim 1 , wherein the scene mode setting comprises sports mode, macro mode, movie mode, night mode, steady shot mode, landscape mode, portrait mode, snow mode, document mode, beach mode, food mode, and/or fireworks mode.

14 . The method of claim 1 , wherein automatically selecting one of the plurality of scene mode settings for the scene based on the digital depth map of the scene comprises:
setting parameters related to shutter speed, aperture, white balance, color saturation, focus, and/or exposure.

15 . A device for operating a camera, comprising:
a computation unit configured to calculate a digital depth map of a scene that is received by the camera; and a selection unit configured to automatically select one of a plurality of scene mode settings for the scene based on the digital depth map of the scene.

16 . The device of claim 15 ,
wherein the selection unit is further configured to precede automatically selecting one of the plurality of scene mode settings by determining an initial scene mode setting out of the plurality of scene mode settings, based on non-depth information related to the scene; and wherein the selection unit is further configured to automatically select one of the plurality of scene mode settings by automatically changing the initial scene mode setting based on the digital depth map of the scene.

17 . The device claim 15 , wherein the selection unit is further configured to:
identify one or more objects in the scene based on the digital depth map; assign depth values to each of the one or more of the objects in the scene; weight the one or more objects in the scene based on the assigned depth values; and automatically select the scene mode setting based on the respective weighting of the one or more objects.

18 . The device of claim 17 , wherein the selection unit is further configured to:
classify one or more of a plurality of pixels in the scene into depth ranges based on the digital depth map; and identify the one or more objects in the scene based on the classification of the one or more of the plurality of pixels in the scene into depth ranges.

19 . The device claim 17 , wherein the selection unit is further configured to:
determine the respective type of respective ones of the one or more objects in the scene;
assign priorities to the one or more objects in the scene that were identified based on the determined respective type of the respective ones of the one or more objects in the scene; and
weight the one or more objects in the scene based on the priorities that were assigned to the one or more objects.

20 . A computer program product for operating a camera, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein, the co

Assignees

• Sony Corp

Inventors

• Linåker Daniel

Classifications

• H04N13/271 Primary

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• H04N5/2226 Primary

  Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10) · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N5/23245

  Electricity · mapped topic