Vehicle-trailer rearview vision system and method

Patent metadata

Field	Value
Publication number	US-10549694-B2
Application number	US-201815889470-A
Country	US
Kind code	B2
Filing date	Feb 6, 2018
Priority date	Feb 6, 2018
Publication date	Feb 4, 2020
Grant date	Feb 4, 2020

Abstract

A method and system for generating a combined rearview image of an area behind a vehicle that is towing a trailer, the method including: capturing a first image from a vehicle camera that is installed on the vehicle, the first image being comprised of an area behind the vehicle; capturing a second image from a trailer camera that is installed on the trailer, the second image being comprised of an area behind the trailer; determining an obstructed viewing region within the first image, the obstructed viewing region being a region of the first image in which the trailer resides; overlaying the second image on the first image and at least partially within the obstructed viewing region; and inserting graphics in the first image, in the second image, and/or in a patch region within the obstructed viewing region and between the second image and the first image.

First claim

The invention claimed is:

1. A method of generating a combined rearview image of an area behind a vehicle that is towing a trailer, the method comprising:
capturing a first image from a vehicle camera that is installed on the vehicle, the first image being comprised of an area behind the vehicle;
capturing a second image from a trailer camera that is installed on the trailer, the second image being comprised of an area behind the trailer;
determining an obstructed viewing region within the first image, the obstructed viewing region being a region of the first image in which the trailer resides;
removing a non-obstructed portion of the first image, wherein the non-obstructed portion is an area of the first image that is not obstructed by the trailer;
overlaying at least a portion of the second image at least partially within the obstructed viewing region of the first image; and
inserting graphics at least in a patch region within the obstructed viewing region and between the second image and the first image, wherein the patch region includes the non-obstructed portion such that at least some of the graphics are inserted into the non-obstructed portion so as to allow a smoother transition to be provided between the first image and the second image that is overlaid at least partially within the obstructed viewing region of the first image.

2. The method of claim 1 , further comprising the step of determining an image overlay area within the first image and/or at least partly overlapping the obstructed viewing region, wherein the second image is overlaid within the image overlay area.

3. The method of claim 2 , further comprising the step of determining a trailer articulation angle, wherein the image overlay area is determined based at least partly on the trailer articulation angle.

4. The method of claim 3 , further comprising the step of processing the second image so that the second image after the processing depicts the area behind the trailer from a perspective having a viewing angle that is the same as a viewing angle of the vehicle camera.

5. The method of claim 4 , wherein the processing step is based at least partly on a trailer camera viewing angle and a vehicle camera viewing angle.

6. The method of claim 1 , wherein at least some of the graphics do not depict or attempt to depict the area behind the vehicle or the area behind the trailer, and wherein the at least some of the graphics highlight one or more roadway features.

7. The method of claim 1 , wherein at least some of the graphics depict or attempt to depict the area behind the vehicle and/or the area behind the trailer.

8. The method of claim 7 , wherein the at least some of the graphics are obtained using scene memorization techniques, wherein the scene memorization techniques include the steps of: retaining at least a portion of the first image and, subsequently, recalling the portion of the first image, and using the recalled portion of the first image to fill in at least a portion of the patch regions within the obstructed viewing region.

9. The method of claim 1 , further comprising the step of displaying the combined rearview image on an electronic display mirror included in the vehicle, wherein the electronic display mirror is a rearview mirror that enables a vehicle operator or a vehicle passenger to view an area behind the vehicle, and wherein the display of the combined rearview image enables the vehicle operator or the vehicle passenger to view the area behind the vehicle and the area behind the trailer without being obstructed by the trailer.

10. A method of generating a combined rearview image of an area behind a vehicle that is towing a trailer, the method comprising:
capturing a first image from a vehicle camera that is installed on the vehicle, the first image being comprised of an area behind the vehicle;
capturing a second image from a trailer camera that is installed on the trailer, the second image being comprised of an area behind the trailer;
defining an obstructed viewing region within the first image, the obstructed viewing region including a region within the first image that includes the trailer;
determining an image overlay area within the obstructed viewing region;
inserting the second image on the first image according to the image overlay area;
identifying a portion of the first image that corresponds to a patch region that is within the obstructed viewing region and outside the image overlay area, wherein at least part of the patch region includes a non-obstructed portion that is not obstructed by the trailer;
rendering graphics in the patch region within the obstructed viewing region and outside the image overlay area, wherein the graphics include natural scene graphics that depict or attempt to depict an area within the obstructed viewing region and outside the second image, and wherein at least some of the graphics are inserted into the non-obstructed portion so as to allow a smoother transition to be provided between the first image and the second image; and
displaying a combined image on an electronic display that is installed in the vehicle, wherein the combined image includes at least part of the first image, at least part of the second image, and the rendered graphics.

11. The method of claim 10 , wherein the method is at least partly carried out by an onboard computer installed in the vehicle, wherein the onboard computer carries out the defining step, the determining step, the inserting step, and the rendering step.

12. The method of claim 11 , further comprising the step of receiving the first image and the second image at the onboard computer, wherein the second image is received via a vehicle-trailer interface.

13. The method of claim 11 , wherein a plurality of iterations of the method are carried out including a first iteration and a second iteration, wherein one or more steps of the first iteration are carried out after one or more steps of the second iteration, wherein the first iteration is completed before the second iteration.

14. The method of claim 11 , wherein a plurality of iterations of the method are carried out including a first iteration and a second iteration, wherein the first iteration includes retaining at least a portion of the first image in memory of the onboard computer during the a first iteration, and wherein the rendering step of the second iteration includes using the retained portion of the first image to render the graphics of the second iteration.

15. The method of claim 14 , wherein the second iteration further comprises the step of obtaining a speed of the vehicle, and wherein the rendering step of the second iteration is at least partially based on the obtained speed of the vehicle.

16. The method of claim 10 , further comprising the step of overlaying graphics on the combined image that direct or attempt to direct a vehicle operator's or a vehicle passenger's attention to a roadway feature.

17. A vehicle rearview vision system for use with a vehicle, comprising:
a vehicle camera installed on the vehicle, wherein the vehicle camera is configured to capture an area behind the vehicle;
an electronic display installed on the vehicle;
an onboard computer installed on the vehicle, wherein the onboard computer includes a processor and a non-transitory computer-readable memory, wherein the memory includes a computer program product that, when executed by the processor, causes the vehicle to carry out a combined image generation process that includes:
obtaining a first image from the vehicle camera, the first image being comprised of the a

Assignees

• Gm Global Tech Operations Llc

Inventors

• Murad Mohannad
• Huntzicker Fred W
• Chung Caroline

Classifications

• B60R2300/8046

  for replacing a rear-view mirror system · CPC title

• B60R1/12

  Mirror assemblies combined with other articles, e.g. clocks · CPC title

• B60R2300/105

  using multiple cameras · CPC title

• B60R1/04

  mounted inside vehicle {(B60R1/025, B60R1/08 take precedence)} · CPC title

• B60R2300/303

  using joined images, e.g. multiple camera images · CPC title