Image generating apparatus, image generating method, and recording medium

Patent metadata

Field	Value
Publication number	US-11056001-B2
Application number	US-201715608021-A
Country	US
Kind code	B2
Filing date	May 30, 2017
Priority date	Jun 7, 2016
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

An image generating apparatus generates an image to be displayed on a display and includes at least one memory and a control circuit. The control circuit acquires a plurality of camera images captured by a plurality of cameras installed in a vehicle, calculates a distance between one of the cameras and a target to be projected in the camera images, detects a position of a light-transmissive object or a reflective object in the camera images, and generates an image from a point of view that is different from points of view of the plurality of camera images by using the plurality of camera images and the distance, the generated image including a predetermined image that is displayed at the position of the light-transmissive object or the reflective object.

First claim

What is claimed is:

1. An image generating apparatus that generates an image to be displayed on a display, the image generating apparatus comprising:
at least one memory; and
a control circuit,
wherein the control circuit
(a) acquires a plurality of camera images captured by a plurality of cameras installed in a vehicle,
(b) calculates a distance between one of the cameras and a target in a three-dimensional space by using the plurality of camera images, the plurality of camera images including images of the target,
(c) detects a first position of a light-transmissive and/or light-reflective object in one of the plurality of camera images, and
(d) generates an image including (i) a first image different from the plurality of camera images captured by the plurality of cameras and (ii) a second image generated from an image of the light-transmissive and/or light-reflective object obtained from one of the plurality of camera images,
wherein the first image is generated from a first point of view that is different from points of view from which the plurality of camera images are generated,
wherein the first image is generated by using at least one of the plurality of camera images and the calculated distance,
wherein the second image is displayed at a second position in the generated image, and the second image is generated from a point of view different from the first point of the view, and
wherein the second position is determined from the first point of view and on the basis of the detected first position.

2. The image generating apparatus according to claim 1 ,
wherein the control circuit further
(e) estimates a distance between the camera and the light-transmissive and/or light-reflective object, and
wherein in the (d), the image is generated by using the plurality of camera images and the estimated distance.

3. The image generating apparatus according to claim 1 ,
wherein in the (a), a polarized camera image captured by a polarization camera that captures an image through a polarization plate is acquired, and
wherein in the (c), a position where brightness of the polarized camera image periodically changes is detected as the first position.

4. The image generating apparatus according to claim 1 ,
wherein the control circuit further
(g) acquires a plurality of infrared camera images captured by a plurality of far-infrared cameras, and
(h) calculates a distance between one of the far-infrared cameras and the target included in the infrared camera images in a three-dimensional space by using the plurality of infrared camera images, and
wherein in the (c), a position where a difference between the distance calculated by using the camera images and the distance calculated by using the infrared camera images is larger than a predetermined threshold is detected as the first position.

5. The image generating apparatus according to claim 1 , wherein in the (c), an image of a frame-shaped object included in the one of the plurality of camera images is recognized, and a region inside the recognized image is detected as the first position.

6. The image generating apparatus according to claim 1 , wherein in the (d), the second image is a part of one of the plurality of camera images.

7. The image generating apparatus according to claim 1 , wherein in the (d), the second image is a largest image of the object included in one of the plurality of camera images.

8. The image generating apparatus according to claim 7 , wherein in the (d), a camera image including a larger area of the light-transmissive and/or light-reflective object is preferentially displayed among the plurality of camera images at positions of a plurality of light-transmissive and/or light-reflective objects in the generated image.

9. The image generating apparatus according to claim 1 ,
wherein the control circuit further
(i) separates a transmitted light-component and a reflected light-component in the part of the one of the plurality of camera images, and
wherein in the (d), the second image has a first transmitted light-component and a first reflected light-component, the first transmitted light-component being generated by giving a first predetermined ratio to the transmitted light-component, and the first reflected light-component being generated by giving a second predetermined ratio to the reflected light-component.

10. The image generating apparatus according to claim 9 , wherein in the (i), a polarized camera image captured by a polarization camera that captures an image through a polarization plate is acquired, and the first transmitted light-component and the first reflected light-component are separated from each other by using the polarized camera image.

11. The image generating apparatus according to claim 9 , wherein in the (d), the first predetermined ratio is received from a user, and the first transmitted light-component and the first reflected light-component are displayed by being assigned weights at the first predetermined ratio received from the user at the position of the light-transmissive object in the generated image.

12. The image generating apparatus according to claim 9 , wherein in the (d), the first transmitted light-component and the first reflected light-component are displayed at the position of the light-transmissive object in the generated image by being assigned weights in such a manner that the weight of the first reflected light-component is lower than the weight of the first transmitted light-component when external light is brighter.

13. The image generating apparatus according to claim 1 , wherein in the (d), if pixels more than or equal to a predetermined ratio are saturated in the camera images, such a message is displayed in the generated image that prompts a driver to see a periphery of the vehicle.

14. The image generating apparatus according to claim 1 ,
wherein the plurality of cameras include a first camera that captures an image of a forward area of the vehicle and a second camera that captures an image of a backward area of the vehicle,
wherein the control circuit further
(j) acquires a heading direction of the vehicle, and
wherein in the (c), the first position is detected by using a camera image obtained by a camera that captures an image in a direction that is same as the heading direction of the vehicle, the camera being selected from the first camera and the second camera.

15. An image generating method for generating an image to be displayed on a display installed in a vehicle, the image generating method comprising:
(a) acquiring a plurality of camera images captured by a plurality of cameras installed in the vehicle,
(b) calculating a distance between one of the cameras and a point in a space to be projected in the camera images by using the plurality of camera images,
(c) detecting a first position of a light-transmissive and/or light-reflective object in one of the plurality of camera images, and
(d) generating an image including (i) a first image different from the plurality of camera images captured by the plurality of cameras and (ii) a second image generated from an image of the light-transmissive and/or light-reflective object obtained from one of the plurality of camera images,
wherein the first image is generated from a first point of view that is different from points of view from which the plurality of camera images are generated,
wherein the first image is generated by using at least one of the plurality of camera images and the calculated distance,
wherein the second image is displayed at a second position in the generated image, and the second image

Assignees

• Panasonic Ip Man Co Ltd

Inventors

• Maruyama Ayako
• Nobori Kunio
• Azuma Takeo
• Sato Satoshi
• Wakai Nobuhiko

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• G08G1/166 Primary

  for active traffic, e.g. moving vehicles, pedestrians, bikes · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• B60R2300/607

  from a bird's eye viewpoint · CPC title

• G08G1/165

  for passive traffic, e.g. including static obstacles, trees · CPC title