Merging of partial images to form an image of surroundings of a mode of transport

Patent metadata

Field	Value
Publication number	US-11087438-B2
Application number	US-201715402624-A
Country	US
Kind code	B2
Filing date	Jan 10, 2017
Priority date	Jul 11, 2014
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

A driver assistance system and a method for merging partial images to form an image of a contiguous surrounding region of a mode of transport are provided. The method includes the following steps: detecting a first partial region of the surrounding region by a first environmental sensor; detecting a second partial region of the surrounding region by a second environmental sensor; generating a first partial image of the first partial region on the basis of a signal from the first environmental sensor; generating a second partial image of the second partial region on the basis of a signal of the second environmental sensor and a multiplicity of virtual sensors for detecting the second partial region; and merging the first partial image and the second partial image along a straight first joining line, wherein the virtual sensors are arranged at positions substantially on a perpendicular to the joining line.

First claim

What is claimed is:

1. A method for merging partial images to form an image of a contiguous environment region of a mode of transport, the method comprising the steps of:
capturing a first partial region of the environment region using a first environmental sensor;
capturing a second partial region of the environment region using a second environmental sensor;
generating a first partial image of the first partial region on the basis of a signal of the first environmental sensor, wherein the first partial image occupies an image plane;
generating a second partial image of the second partial region from a signal of the second environmental sensor,
wherein the second partial image occupies the image plane,
wherein the second partial image is generated as a composite of a plurality of virtual image segments occupying the image plane, each virtual image segment derived from the signal of the second environmental sensor and corresponding to a respective virtual perspective of a virtual sensor of a multiplicity of virtual sensors arranged at linearly displaced positions substantially along a line that is perpendicular to the first seam line and parallel to the image plane,
wherein a first outermost virtual sensor of the multiplicity of virtual sensors and the first environmental sensor are arranged substantially along the first seam line; and

merging the first partial image and the second partial image along a straight first seam line.

2. The method as claimed in claim 1 , wherein a virtual sensor is arranged in a direct vicinity of the first seam line.

3. The method as claimed in claim 1 , wherein the virtual sensors are arranged horizontally with respect to one another and spaced apart.

4. The method as claimed in claim 1 , wherein the second partial image comprises segments which are assigned to a virtual sensor, which segments are merged together along second seam lines which extend substantially parallel to the first seam line.

5. The method as claimed in claim 1 , wherein the first environmental sensor is a 2D sensor, and/or the second environmental sensor is a 3D sensor arrangement.

6. The method as claimed in claim 1 , wherein the environmental sensors comprise optical sensors.

7. The method as claimed in claim 1 , further comprising the steps of:
capturing a third partial region of the environment region using a third environmental sensor;
generating a third partial image of the third partial region on the basis of a signal of the third environmental sensor; and
merging the third partial image and the second partial image along a straight third seam line, wherein the first seam line and the third seam line represent mutually opposite boundary lines of the second partial image.

8. The method as claimed in claim 7 , further comprising the steps of:
removing a region of the first partial image, which is potentially located beyond the first seam line, before merging;
removing regions of the second partial image, which are potentially located beyond the first seam line and/or the third seam line, before merging; and/or
removing a region of the third partial image, which is potentially located beyond the third seam line, before merging.

9. The method as claimed in claim 7 , wherein a perspective of a second outermost virtual sensor of the multiplicity of virtual sensors is substantially the same as a perspective of the third environmental sensor.

10. The method as claimed in claim 1 , further comprising the steps of:
generating a first 2D data set from the first partial image with respect to a virtual first projection area, and/or
generating a second 2D data set from the second partial image with respect to a virtual second projection area,
wherein a position of an environment object relative to the mode of transport dynamically determines a position of the projection area.

11. The method as claimed in claim 10 , wherein the first virtual projection area at the first seam line corresponds to the second virtual projection area.

12. The method as claimed in claim 11 , wherein
the first virtual projection area has a concave shape in the manner of a cylindrical segment, and/or
the second virtual projection area has a concave shape in the manner of a cylindrical segment, wherein the respective shapes in the manner of a cylindrical segment belong to a common cylinder.

13. The method as claimed in claim 10 , wherein
the first virtual projection area has a concave shape in the manner of a cylindrical segment, and/or
the second virtual projection area has a concave shape in the manner of a cylindrical segment, wherein the respective shapes in the manner of a cylindrical segment belong to a common cylinder.

14. The method as claimed in claim 1 , further comprising the step of: displaying a portion of an image on a 2D display unit within the mode of transport, where the image comprises a result of said merging the first partial image and the second partial image, and wherein the portion of the image represents captured imagery spanning an angular range of between 180° and 45°.

15. The method as claimed in claim 14 , further comprising the step of:
varying a position of the portion of the image in response to a driving situation which is identified by way of activating an indicator and/or by a preparation for driving in reverse.

16. The method as claimed in claim 1 , further comprising the step of: displaying a portion of an image on a 2D display unit within the mode of transport, where the image comprises a result of said merging the first partial image and the second partial image, and wherein the portion of the image represents captured imagery spanning an angular range of between 120° and 60°.

17. The method as claimed in claim 1 , further comprising the step of: displaying a portion of an image on a 2D display unit within the mode of transport, where the image comprises a result of said merging the first partial image and the second partial image, and wherein the portion of the image represents captured imagery spanning an angular range of between 100° and 80°.

18. A driver assistance system, comprising:
a first environmental sensor;
a second environmental sensor;
an evaluation unit; and
a 2D display unit, wherein the driver assistance system is adapted to carry out a method as claimed in claim 1 .

19. A mode of transport, comprising:
a driver assistance system, comprising:
a first environmental sensor;
a second environmental sensor;
an evaluation unit; and
a 2D display unit, wherein the driver assistance system is adapted to carry out a method as claimed in claim 1 .

20. The mode of transport as claimed in claim 19 , wherein the mode of transport is a passenger vehicle, a transporter, a truck, a watercraft and/or aircraft.

Assignees

• Bayerische Motoren Werke Ag

Inventors

• Schmidt Torsten
• Hoffmann Philipp

Classifications

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T2200/32

  involving image mosaicing · CPC title