Systems and method for image normalization

Patent metadata

Field	Value
Publication number	US-12225319-B2
Application number	US-201917425028-A
Country	US
Kind code	B2
Filing date	Feb 15, 2019
Priority date	Feb 15, 2019
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

A system for producing a virtual image view for a vehicle is provided. The system includes one or more image capture means configured to capture image data in proximity to the vehicle, the image data being defined at least in part by first viewpoint parameters, and to provide an identifier identifying the respective one or more image capture means, storage means configured to store a plurality of virtualization records containing conversion information related to a virtualized viewpoint and a plurality of image capture means, and processing means. The processing means are configured to receive the captured image data, convert the first viewpoint parameters of the captured image data into virtual viewpoint parameters based on the conversion information associated with a virtualization record stored by the storage means, to result in the virtual image view, wherein the virtualization record is identified at least based on the identifier, and execute at least one driver assistance and/or automated driving function based on the virtual image view.

First claim

The invention claimed is:

1. A system for producing a virtual image view for a vehicle, comprising:
one or more image capture means configured to capture image data in proximity to the vehicle, the image data being defined at least in part by first viewpoint parameters, and to provide an identifier identifying the respective one or more image capture means;
storage means configured to store a plurality of virtualization records containing conversion information related to a virtualized viewpoint, a plurality of image capture means, and to a plurality of processing applications comprising at least two among an object detection and tracking application and a lane identifier application; and
processing means configured to:
receive the captured image data;
identify, among the plurality of processing applications, a processing application expecting virtual viewpoint parameters based on the captured image data;
select the processing application expecting virtual viewpoint parameters;
convert the first viewpoint parameters of the captured image data into the virtual viewpoint parameters of the selected processing application based on the conversion information associated with a virtualization record stored by the storage means, to result in the virtual image view, wherein the virtualization record is identified at least based on the identifier and on the identified processing application; and
execute at least one driver assistance and/or automated driving function based on the virtual image view.

2. The system for producing a virtual image view for a vehicle according to claim 1 , wherein the conversion information comprises at least one of distortion compensation information, image rectification information, image refraction information, and rotational information.

3. The system for producing a virtual image view according to claim 1 , wherein each virtualized viewpoint is defined by at least one of a resolution and a field of view; and
at least one of a pinhole model, a fisheye model, a cylindrical model, a spherical model, and a rectilinear model.

4. The system for producing a virtual image view according to claim 1 , wherein the storage means comprises means for identifying the processing application linked to a virtual image view.

5. The system for producing a virtual image view according to claim 4 , wherein the storage means comprises a lookup table linking a camera identifier with conversion information for generation of the virtual image view required by the processing application.

6. The system for producing a virtual image view according to claim 1 , wherein the conversion information includes calibration parameters for an identified image capture means, and wherein the processing means is configured to verify the calibration parameters for each of the one or more image capture means.

7. A vehicle comprising the system according to claim 1 .

8. A method for producing a virtual image view for a vehicle, comprising:
receiving image data captured by at least one image capture means from surroundings of the vehicle, the image data being defined at least in part by first viewpoint parameters;
receiving an identifier identifying one or more image capture means providing the image data;
identifying, among a plurality of processing applications comprising at least two among an object detection and tracking application and a lane identifier application, a processing application expecting viewpoint parameters;
selecting the processing application expecting viewpoint parameters;
determining a virtualization record among the virtualization records containing conversion information related to a virtualized viewpoint, a plurality of image capture means, and the plurality of processing applications, wherein the virtualization record is determined at least based on the identifier and the identified processing application;
converting the first viewpoint parameters of the captured image data into the virtual viewpoint parameters of the selected processing application based on the conversion information associated with the determined virtualization record and the identified processing application, to result in the virtual image view; and
executing at least one driver assistance and/or automated driving function based on the virtual image view.

9. The method for producing a virtual image view for a vehicle according to claim 8 , wherein the conversion information comprises at least one of distortion compensation information, image rectification information, image refraction information, and rotational information.

10. The method for producing a virtual image view according to claim 8 , wherein each virtualized viewpoint is defined by at least one of a resolution and a field of view; and
at least one of a pinhole model, a fisheye model, a cylindrical model, a spherical model, and a rectilinear model.

11. The method for producing a virtual image view according to claim 8 , wherein the determining includes identifying the processing application linked to a virtual image view.

12. The method for producing a virtual image view according to claim 11 , wherein the determining includes searching a lookup table, the lookup table including information linking a camera identifier with conversion information for generation of the virtual image view required by the processing application.

13. The method for producing a virtual image view according to claim 8 , wherein the conversion information includes calibration parameters for an identified image capture means, and the method further comprising verifying the calibration parameters for each of the one or more image capture means.

Assignees

• Toyota Motor Europe
• Univ Leuven Kath
• Toyota Motor Co Ltd

Inventors

• Tamura Kazuki
• Shimizu Hiroaki
• Proesmans Marc
• Verbiest Frank
• Van Gool Luc

Classifications

• B60K35/28

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title

• B60K35/22

  Display screens · CPC title

• B60K35/60

  Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10) · CPC title

• B60K35/10

  Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor · CPC title

• B60K2360/21

  using cameras · CPC title