Conditional availability of vehicular mixed-reality

Patent metadata

Field	Value
Publication number	US-10607416-B2
Application number	US-201816118018-A
Country	US
Kind code	B2
Filing date	Aug 30, 2018
Priority date	Aug 30, 2018
Publication date	Mar 31, 2020
Grant date	Mar 31, 2020

Abstract

Methods, apparatuses, and computer-readable media are presented for providing a mixed-reality scene involving a lead vehicle and a following vehicle. The system may present a sequence of mixed-reality images to a driver of the following vehicle, resulting from merging (a) an image captured by a camera aboard the lead vehicle and (b) an image captured by a camera aboard the following vehicle, to generate a merged image. The system may discontinue or diminish mixed-reality content of the sequence of mixed-reality images in response to one or more detected conditions, which may comprise detection of an object between the lead vehicle and the following vehicle.

First claim

What is claimed is:

1. A method for providing a mixed-reality scene involving a lead vehicle and a following vehicle comprising:
presenting a sequence of mixed-reality images to a driver of the following vehicle, wherein at least one image in the sequence of mixed-reality images results from merging (a) an image captured by a camera aboard the lead vehicle and (b) an image captured by a camera aboard the following vehicle, to generate a merged image;
wherein the merging comprises de-emphasizing an occluded portion of the image captured by the camera aboard the following vehicle, the occluded portion corresponding to occlusion by the lead vehicle, and emphasizing an unoccluded portion of the image captured by the camera aboard the lead vehicle;
in response to one or more detected conditions, discontinuing or diminishing mixed-reality content of the sequence of mixed-reality images presented to the driver of the following vehicle, wherein the one or more detected conditions comprise:
detection of an object between the lead vehicle and the following vehicle, wherein in the merged image, a view of the object is at least partially masked as result of de-emphasizing the occluded portion of the image captured by the camera aboard the following vehicle and emphasizing the unoccluded portion of the image captured by the camera aboard the lead vehicle.

2. The method of claim 1 , wherein the de-emphasizing of the occluded portion of the image captured by the camera aboard the following vehicle and the emphasizing of the unoccluded portion of the image captured by the lead vehicle comprise:
blending the image captured by the camera aboard the following vehicle and the image captured by the camera aboard the lead vehicle.

3. The method of claim 1 , wherein the de-emphasizing of the occluded portion of the image captured by the camera aboard the following vehicle and the emphasizing of the unoccluded portion of the image captured by the lead vehicle comprise:
replacing the occluded portion of the image captured by the camera aboard the following vehicle with the unoccluded portion of the image captured by the lead vehicle.

4. The method of claim 1 , wherein the diminishing of the mixed-reality content of the sequence of mixed-reality images comprises emphasizing the occluded portion of the image captured by the camera aboard the following vehicle and de-emphasizing the unoccluded portion of the image captured by the camera aboard the lead vehicle.

5. The method of claim 1 , wherein the one or more detected conditions further comprise detection of a braking condition associated with the lead vehicle.

6. The method of claim 1 ,
wherein the one or more detected conditions further comprise detection of at least one relevant maneuver performed by the following vehicle,
wherein the at least one relevant maneuver is selected from an over-taking maneuver, a right turn, a mid-block pedestrian crossing event, or an unexpected lead vehicle stop.

7. The method of claim 1 , wherein the one or more detected conditions further comprise detection of improper camera alignment associated with the camera aboard the lead vehicle and the camera aboard the following vehicle.

8. The method of claim 1 , wherein the one or more detected conditions further comprise determination that a candidate lead vehicle is not positioned in front of the following vehicle.

9. The method of claim 1 , wherein discontinuing or diminishing mixed-reality content of the sequence of mixed-reality images comprises:
in the merged image, presenting a representation of the object between the lead vehicle and the following vehicle.

10. The method of claim 9 , wherein the representation of the object between the lead vehicle and the following vehicle is presented by:
defining a region in the merged image containing the object between the lead vehicle and the following vehicle; and
in the defined region, presenting the representation of the object between the lead vehicle and the following vehicle instead of the unoccluded portion of the image captured by the lead vehicle.

11. An apparatus for providing a mixed-reality scene involving a lead vehicle and a following vehicle comprising:
an electronic control unit (ECU); and
a display,
wherein the ECU is configured to:
generate a sequence of mixed-reality images for presentation to a driver of the following vehicle, wherein the ECU is configured to generate at least one image in the sequence of mixed-reality images results by merging (a) an image captured by a camera aboard the lead vehicle and (b) an image captured by a camera aboard the following vehicle, to generate a merged image;
wherein the merging comprises de-emphasizing an occluded portion of the image captured by the camera aboard the following vehicle, the occluded portion corresponding to occlusion by the lead vehicle, and emphasizing an unoccluded portion of the image captured by the camera aboard the lead vehicle;
in response to one or more detected conditions, discontinue or diminish mixed-reality content of the sequence of mixed-reality images presented to the driver of the following vehicle, wherein the one or more detected conditions comprise:
detection of an object between the lead vehicle and the following vehicle, wherein in the merged image, a view of the object is at least partially masked as result of de-emphasizing the occluded portion of the image captured by the camera aboard the following vehicle and emphasizing the unoccluded portion of the image captured by the camera aboard the lead vehicle, and

wherein the display is configured to present the sequence of mixed-reality images to the driver of the following vehicle.

12. The apparatus of claim 11 , wherein the ECU is configured to de-emphasize the occluded portion of the image captured by the camera aboard the following vehicle and emphasize the unoccluded portion of the image captured by the lead vehicle by:
blending the image captured by the camera aboard the following vehicle and the image captured by the camera aboard the lead vehicle.

13. The apparatus of claim 11 , wherein the ECU is configured to de-emphasize the occluded portion of the image captured by the camera aboard the following vehicle and the emphasize the unoccluded portion of the image captured by the lead vehicle by:
replacing the occluded portion of the image captured by the camera aboard the following vehicle with the unoccluded portion of the image captured by the lead vehicle.

14. The apparatus of claim 11 , wherein the ECU is configured to diminish the mixed-reality content of the sequence of mixed-reality images by emphasizing the occluded portion of the image captured by the camera aboard the following vehicle and de-emphasizing the unoccluded portion of the image captured by the camera aboard the lead vehicle.

15. The apparatus of claim 11 , wherein the one or more detected conditions further comprise detection of a braking condition associated with the lead vehicle.

16. The apparatus of claim 11 , wherein the one or more detected conditions further comprise detection of at least one relevant maneuver performed by the following vehicle.

17. The apparatus of claim 11 , wherein the one or more detected conditions further comprise detection of improper camera alignment associated with the camera aboard the lead vehicle and the camera aboard the following vehicle.

18. The apparatus of claim 11 , wherein the ECU is configured to discontinue or diminish mixed-reality content of the sequence of mixed-reality images by:
in the merged image, presenting a

Assignees

• Valeo Comfort & Driving Assistance

Inventors

• Nowakowski Christopher Steven
• Kuoch Siav-Kuong
• Garnault Alexandre Jacques
• Abuelfoutouh Mohamed Amr Mohamed Nader

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• B60R2300/802

  for monitoring and displaying vehicle exterior blind spot views · CPC title

• B60R2300/105

  using multiple cameras · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• B60R2300/303

  using joined images, e.g. multiple camera images · CPC title