Electronic device and notification method

Patent metadata

Field	Value
Publication number	US-11418707-B2
Application number	US-201916292438-A
Country	US
Kind code	B2
Filing date	Mar 5, 2019
Priority date	Sep 18, 2018
Publication date	Aug 16, 2022
Grant date	Aug 16, 2022

Abstract

According to one embodiment, an electronic device includes one or more processors. The one or more processors obtain an image captured by a camera with a filter having a first area transmitting light of a first wavelength range and a second area transmitting light of a second wavelength range. The image includes a first color-component image based on the light of the first wavelength range and a second color-component image based on the light of the second wavelength range. The one or more processors notify a user of an effective area for calculation of depth information based on a bias of color information in the first color-component image and the second color-component image.

First claim

What is claimed is:

1. An electronic device comprising:
one or more processors configured to
obtain an image captured by a camera comprising a lens, a sensor, and a filter, the filter having a first area transmitting light of a first wavelength range and a second area transmitting light of a second wavelength range, the first wavelength range related to a first color, the second wavelength range related to a second color, the filter arranged outside the lens, the sensor capturing a first color-component image based on the light of the first wavelength range transmitted through the first area and a second color-component image based on the light of the second wavelength range transmitted through the second area,
determine a color shift in the first color-component image and the second color-component image;
determine an effective area on the image for calculation of depth information based on the color shift, such that the reliability of the depth information within the effective area is higher than a reference reliability; and
notify a user of the effective area.

2. The electronic device according to claim 1 , wherein
the first wavelength range is further related to a third color;
the second wavelength range is further related to the third color;
the sensor further captures a third-color component image based on the light of the first wavelength range transmitted through the first area and the light of the second wavelength range transmitted through the second area; and
the depth information is obtained based on a shape of a blur of the first color-component image, a shape of a blur of the second color-component image, and a shape of a blur of the third color-component image.

3. The electronic device according to claim 1 , wherein
the one or more processors are configured to acquire or calculate the effective area based on at least one capturing parameter among plural capturing parameters.

4. The electronic device according to claim 3 , wherein
the one or more processors are configured to transmit an effective area acquisition request including the at least one capturing parameter to an external device, and receive the effective area corresponding to the at least one capturing parameter from the external device.

5. The electronic device according to claim 3 , further comprising:
a memory that stores the effective area obtained from a result of simulation of an optical characteristic distribution of the camera based on the at least one capturing parameter.

6. The electronic device according to claim 3 , wherein
the plural capturing parameters comprise a distance between an aperture of the lens and the filter, a sensor size of the camera, a focal length of the lens, or a brightness of the lens.

7. The electronic device according to claim 4 , wherein
the plural capturing parameters comprise a distance between an aperture of the lens and the filter, a sensor size of the camera, a focal length of the lens, or a brightness of the lens.

8. The electronic device according to claim 5 , wherein
the plural capturing parameters comprise a distance between an aperture of the lens and the filter, a sensor size of the camera, a focal length of the lens, or a brightness of the lens.

9. The electronic device according to claim 1 , wherein
the filter is arranged at a front end of the lens or to be closer to an object than the front end, or is arranged at a rear end of the lens or to be closer to an image sensor than the rear end.

10. The electronic device according to claim 9 , wherein
when an object for which depth information in the image is to be obtained is designated, the one or more processors are configured to detect a direction of an edge component of the object and notify information corresponding to a detected direction of the edge component.

11. The electronic device according to claim 2 , wherein
the filter is arranged at a front end of the lens or to be closer to an object than the front end, or is arranged at a rear end of the lens or to be closer to an image sensor than the rear end.

12. The electronic device according to claim 3 , wherein
the filter is arranged at a front end of the lens or to be closer to an object than the front end, or is arranged at a rear end of the lens or to be closer to an image sensor than the rear end.

13. The electronic device according to claim 1 , wherein
the one or more processors are configured to extract an object within the effective area from among objects in the image obtained by the one or more processors, and display an extracted object within the effective area.

14. The electronic device according to claim 1 , wherein
the one or more processors are configured to notify guidance for capturing such that a measurement target for which depth information in the image is to be obtained is positioned within the effective area using at least any of a voice, text, or an image.

15. The electronic device according to claim 1 , wherein
the one or more processors are configured to specify a measurement target for which depth information in the image is to be obtained by any of a user operation, a distribution of depth information within the effective area, or object recognition from the image.

16. The electronic device according to claim 1 , wherein
the one or more processors are configured to notify a guidance to recommend re-capturing when determining that a measurement target for which depth information in the image is to be obtained is not within the effective area.

17. The electronic device according to claim 1 , wherein
the one or more processors are configured to display a guidance of the effective area on the image using at least any of a color, brightness, or contrast.

18. The electronic device according to claim 1 , further comprising:
a display device that displays the image obtained by the one or more processors,
wherein the one or more processors are configured to correct the color shift of the image in accordance with the effective area, and
the display device displays the image in which the color shift is corrected.

19. The electronic device according to claim 1 , further comprising:
a display device that displays the image obtained by the one or more processors with at least one of the effective area or reliability information relating to the reliability of the depth estimation used to obtain the effective area.

20. A notification method comprising:
inputting an image captured by a camera comprising a lens, a sensor, and a filter, the filter having a first area transmitting light of a first wavelength range and a second area transmitting light of a second wavelength range, the first wavelength range related to a first color, the second wavelength range related to a second color, the filter arranged outside the lens, the sensor capturing a first color-component image based on the light of the first wavelength range transmitted through the first area and a second color-component image based on the light of the second wavelength range transmitted through the second area;
determining a color shift in the first color-component image and the second color-component image;
determining an effective area on the image for calculation of depth information based on the color shift, such that the reliability of the depth information within the effective area is higher than a reference reliability; and
notifying a user of the effective area.

Assignees

• Toshiba Kk

Inventors

• Kozakaya Tatsuo
• Mishima Nao
• Moriya Akihisa
• Yamaguchi Jun
• Tsuruyama Tomoya
• Sasaki Takayuki

Classifications

• H04N13/271 Primary

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• G06V10/10

  Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60) · CPC title

• G06V10/24

  Aligning, centring, orientation detection or correction of the image · CPC title

• G06V10/147

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title

• G06V10/143

  Sensing or illuminating at different wavelengths · CPC title