Advanced blending of stitched images for 3D object reproduction

Patent metadata

Field	Value
Publication number	US-10055643-B2
Application number	US-201414491024-A
Country	US
Kind code	B2
Filing date	Sep 19, 2014
Priority date	Sep 19, 2014
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

When blending individual vehicle-mounted camera images into a coherent composite image of the vehicle surroundings, objects in the respective images are analyzed to determine the presence of a pedestrian. When the individual images are blended to form the composite image, multiple pedestrian shadow views of the pedestrian are replaced with background pixels to erase the pedestrian, and a semi-symbolic graphical representation of the pedestrian is superimposed at the location of the actual pedestrian in the composite image. The graphical representation is proportional to the pedestrian in the composite image and includes features unique to the pedestrian it represents. The composite image with the graphical representation is then presented to a viewer.

First claim

Having thus described the preferred embodiments, the invention is now claimed to be:

1. A system that facilitates detecting a person in a surround-view image of a vehicle and superimposing a semi-symbolic graphical representation of the person in the surround-view image, comprising:
a first camera that captures a first image of a person from a first vantage point;
a second camera that captures a second image of the person from a second vantage point;
a processor configured to:
generate a composite image by merging the first and second images;
analyze the composite image to detect one or more features indicative of the person in the composite image;
replace pixels corresponding to the person with corresponding background pixels;
generate a height-proportional semi-symbolic graphical representation of the person that comprises one or more features corresponding to the person and detected in the first and second images;
insert the semi-symbolic graphical representation of the person into the composite image at a calculated position of the person relative to the vehicle; and
output the composite image with the semi-symbolic representation to a display.

2. The system according to claim 1 , wherein the processor is further configured to replace pixels corresponding to the person in the first image with background pixels from the second image, and replace pixels corresponding to the person in the second image with background pixels from the first image.

3. The system according to claim 1 , wherein the person is a pedestrian and the one or more features comprise one or more of:
pedestrian clothing color;
and
pedestrian height in pixels.

4. The system according to claim 1 , wherein the person is a pedestrian and the processor is further configured to superimpose an image of the pedestrian's face, or a similar face, on the semi-symbolic graphical representation.

5. The system according to claim 1 , wherein the person is a pedestrian and the processor is further configured to execute a shape identification module that identifies a shape formed by each of a front view representation and a rear view representation of the pedestrian in the composite image, wherein the position of the pedestrian relative at the vehicle is determined to be at the point of intersection of the front and rear view representations of the pedestrian in the composite image.

6. The system according to claim 5 , wherein the person is a pedestrian and the processor is further configured to determine a height of the head of the pedestrian above the ground in the composite image, and wherein the processor is configured to execute a tracking filter that tracks the location of the head of the pedestrian and to adjust the position of the semi-symbolic graphical representation to correspond to the location of the head of the pedestrian.

7. A method of detecting a person in a surround-view image of a vehicle and superimposing a semi-symbolic graphical representation of the person in the surround-view image, comprising:
capturing a first image, from a first vantage point, of a person;
capturing a second image, from a second vantage point, of the person;
generating a composite image by merging the first and second images;
analyzing the composite image to detect one or more features indicative of the person in the composite image;
replacing pixels corresponding to the person with corresponding background pixels;
generating a height-proportional semi-symbolic graphical representation of the person that comprises one or more features corresponding to the person and detected in the first and second images;
inserting the semi-symbolic graphical representation of the person into the composite image at a calculated position of the person relative to the vehicle; and
outputting the composite image with the semi-symbolic representation to a display.

8. The method according to claim 7 , further comprising replacing replace pixels corresponding to the person in the first image with background pixels from the second image, and replace pixels corresponding to the person in the second image with background pixels from the first image.

9. The method according to claim 7 , wherein the person is a pedestrian and the one or more features comprise one or more of:
pedestrian clothing color; and
pedestrian height in pixels.

10. The method according to claim 7 , the person is a pedestrian and further comprising superimposing an image of the pedestrian's face on the semi-symbolic graphical representation.

11. The method according to claim 7 , wherein the person is a pedestrian and further comprising identifying a shape formed by each of a front view representation and a rear view representation of the pedestrian in the composite image, wherein the pedestrian is determined to be positioned where the front and rear view representations intersect in the composite image.

12. The method according to claim 11 , further comprising determining a height of the head of the pedestrian above the ground in the composite image, and tracking the location of the head of the pedestrian as it moves past the vehicle.

13. A processor configured to execute computer-executable instructions for detecting a person in a surround-view image of a vehicle and superimposing a semi-symbolic graphical representation of the person in the surround-view image, the instructions comprising:
capturing a first image, from a first vantage point, of a person;
capturing a second image, from a second vantage point, of the person;
generating a composite image by merging the first and second images;
analyzing the composite image to detect one or more features indicative of the person in the composite image;
replacing pixels corresponding to the person with corresponding background pixels;
generating a height-proportional semi-symbolic graphical representation of the person that comprises one or more object features detected in the first and second images;
inserting the semi-symbolic graphical representation of the person into the composite image at a calculated position of the person relative to the vehicle; and
outputting the composite image with the semi-symbolic representation to a display.

14. The processor according to claim 13 , the instructions further comprising replacing pixels corresponding to the person in the first image with background pixels from the second image, and replace pixels corresponding to the person in the second image with background pixels from the first image.

15. The processor according to claim 13 , wherein the person is a pedestrian and the one or more features comprise one or more of:
pedestrian clothing color;
and
pedestrian height in pixels.

16. The processor according to claim 13 , wherein the person is a pedestrian, the instructions further comprising superimposing an image of the pedestrian's face on the semi-symbolic graphical representation.

17. The processor according to claim 13 , wherein the person is a pedestrian, the instructions further comprising identifying a shape formed by each of a front view representation and a rear view representation of the pedestrian in the composite image, wherein the pedestrian is determined to be positioned where the front and rear view representations intersect in the composite image.

18. The processor according to claim 17 , wherein the person is a pedestrian, the instructions further comprising determining a height of the head of the pedestrian above the ground in the composite image, and tracking the location of the head of the pedestrian as it moves pa

Assignees

• Molin Hans M
• Kuehnle Andreas U
• Gyori Marton
• Bendix Commercial Vehicle Systems Llc

Inventors

• Molin Hans M
• Kuehnle Andreas U
• Gyori Marton

Classifications

• H04N13/239 Primary

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06T7/174

  involving the use of two or more images · CPC title

• G06K9/00671 Primary

  Physics · mapped topic

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title