Model-based method for 360 degree surround view using cameras and radars mounted around a vehicle

Patent metadata

Field	Value
Publication number	US-10864860-B2
Application number	US-201816044203-A
Country	US
Kind code	B2
Filing date	Jul 24, 2018
Priority date	Jul 24, 2018
Publication date	Dec 15, 2020
Grant date	Dec 15, 2020

Abstract

A method of stitching panoramic optical images around a vehicle having, receiving a plurality of optical images from a plurality of image sensors mounted on the vehicle of a plurality of objects surrounding the vehicle, overlapping optical images of the plurality of objects surrounding the vehicle, identifying seams between the plurality of objects surrounding the vehicle within the plurality of optical images, receiving a plurality of radar images from a plurality of radar sensors mounted on the vehicle indicating at least one direction and at least one distance of at least one of the plurality objects surrounding the vehicle, linking the plurality of optical images of the plurality of objects surrounding the its vehicle to the plurality of radar images and stitching the plurality of optical images based on the plurality of radar images.

First claim

What is claimed is:

1. A method of stitching panoramic optical images around a vehicle, comprising:
receiving a plurality of optical images from a plurality of image sensors mounted on the vehicle of a plurality of objects surrounding the vehicle;
receiving a plurality of radar images from a plurality of radar sensors mounted on the vehicle indicating at least one direction and at least one distance of at least one of the plurality objects surrounding the vehicle at a plurality of directions;
fitting the plurality of radar images to a two dimensional curve fit surrounding the vehicle;
linking the plurality of optical images of the plurality of objects surrounding the vehicle to the plurality of radar images;
overlapping optical images of the plurality of objects surrounding the vehicle;
identifying seams between the plurality of objects surrounding the vehicle within the plurality of optical images, wherein the seams are placed through areas lacking texture; and
stitching the plurality of optical images based on the plurality of radar images at the seams.

2. The method of stitching panoramic optical images of claim 1 , wherein the stitching removes redundant optical images.

3. The method of stitching panoramic optical images of claim 1 , further comprising constructing a three dimensional mesh model from a ground surrounding the vehicle and having a mesh model outline following the radar two dimensional curve fit.

4. The method of stitching panoramic optical images of claim 1 , further comprising constructing a three dimensional mesh model from a around surrounding the vehicle and having a mesh model contour following the radar two dimensional curve fit.

5. The method of stitching panoramic optical images of claim 3 , further comprising projecting the plurality of optical images onto the three dimensional mesh model.

6. The method of stitching panoramic optical images of claim 5 , further comprising identifying seams having a minimal texture between the projected plurality of optical images.

7. The method of stitching panoramic optical images of claim 5 , further comprising blending the identified seams.

8. A method of stitching a 360 degree optical image, comprising:
receiving a plurality of optical images of a plurality of surrounding objects, wherein the plurality of optical images comprises the 360 degree optical image;
receiving a plurality of radar images indicating at least one direction and at least one distance of at least one of the plurality surrounding objects at a plurality of directions;
fitting the plurality of radar images to a two dimensional curve fit surrounding the vehicle;
linking the plurality of optical images of the plurality of surrounding objects to the plurality of radar images;
overlapping optical images of the plurality of surrounding objects within the plurality of optical images;
identifying seams between the plurality of surrounding objects within the plurality of optical images, wherein the seams are placed through areas lacking texture; and
stitching the plurality of optical images based on the plurality of radar images at the seams.

9. The method of stitching the 360 degree optical image of claim 8 , further comprising constructing a three dimensional mesh model from a ground level and having a mesh model outline following the radar two dimensional curve fit.

10. The method of stitching the 360 degree optical image of claim 9 , further comprising projecting the plurality of optical images onto the three dimensional mesh model.

11. The method of stitching the 360 degree optical image of claim 10 , further comprising identifying seams having a minimal texture between the projected plurality of optical images.

12. The method of stitching the 360 degree optical image of claim 11 , further comprising blending the identified seams.

13. A method of stitching panoramic optical images, comprising:
receiving a plurality of optical images of a plurality of surrounding objects, wherein the plurality of optical images comprises a panoramic image;
overlapping optical images of the plurality of surrounding objects within the plurality of optical images;
identifying seams between the plurality of surrounding objects within the plurality of optical images, wherein the seams are placed through areas lacking texture;
receiving a plurality of radar images indicating at least one direction and at least one distance of at least one of the plurality surrounding objects at a plurality of directions;
fitting the plurality of radar images to a two dimensional curve fit surrounding the vehicle;
linking the plurality of optical images of the plurality of surrounding objects to the plurality of radar images; and
stitching the plurality of optical images based on the plurality of radar images at the seams.

14. The method of stitching panoramic optical images of claim 13 , further comprising:
fitting plurality of radar images is a two dimensional curve fit of the plurality of surrounding objects;
constructing a three dimensional mesh model from a ground level and having a mesh model outline following the radar two dimensional curve fit; and
projecting the plurality of optical images onto the three dimensional mesh model.

15. The method of stitching panoramic optical images of claim 14 , further comprising:
identifying seams having a minimal texture between the projected plurality of optical images; and
blending the identified seams.

Assignees

• Black Sesame International Holding Ltd

Inventors

• Li Jin

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G01S13/89

  for mapping or imaging · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title