System and method for estimating vehicle dynamics using feature points in images from multiple cameras

Patent metadata

Field	Value
Publication number	US-9619894-B2
Application number	US-201514710364-A
Country	US
Kind code	B2
Filing date	May 12, 2015
Priority date	May 16, 2014
Publication date	Apr 11, 2017
Grant date	Apr 11, 2017

Abstract

A system and method for estimating dynamics of a mobile platform by matching feature points in overlapping images from cameras on the platform, such as cameras in a surround-view camera system on a vehicle. The method includes identifying overlap image areas for any two cameras in the surround-view camera system, identifying common feature points in the overlap image areas, and determining that the common feature points in the overlap image areas are not at the same location. The method also includes estimating three-degree of freedom vehicle dynamic parameters from the matching between the common feature points, and estimating vehicle dynamics of one or more of pitch, roll and height variation using the vehicle dynamic parameters.

First claim

What is claimed is:

1. A method for estimating vehicle dynamics using images from cameras in a surround-view camera system on a vehicle, said method comprising:
providing digital images from the cameras to a processor;
identifying one or more overlap image areas for any two cameras in the surround-view camera system;
identifying, using the processor, common feature points in the one or more overlap image areas;
determining, using the processor, that the identified common feature points are not at the same location in the overlap image areas;
estimating, using the processor, three-degree of freedom vehicle dynamic parameters from comparisons between the common feature points; and
estimating, using the processor, vehicle dynamics of one or more of pitch, roll and height variation using the vehicle dynamic parameters.

2. The method according to claim 1 wherein identifying common feature points includes identifying a region of interest in the overlap image areas that includes the common feature points.

3. The method according to claim 1 wherein identifying common feature points includes identifying common feature points in the images from two cameras that are projections of the same point location in world coordinates.

4. The method according to claim 1 wherein estimating vehicle dynamics includes solving dynamic equations to obtain an estimate of the pitch, roll and/or height variation of the vehicle based on a difference between the common feature points.

5. The method according to claim 1 wherein estimating vehicle dynamics includes determining a rotation matrix and a translation vector.

6. The method according to claim 5 wherein estimating vehicle dynamics includes using the equations:
( u,v ) C1 ˜( u,v ) C2 ,

( u,v ) C1 →X C1 ,
( u,v ) C2 →X C2 ,
X C1 =R V2C1 *R dyn *( X W +t′ W2V )+ t′ V2C1 ,
X C2 =R V2C2 *R dyn *( X W +t′ W2V )+ t′ V2C2 ,

where (u,v) C1 is one feature point in an image from a first camera c1 that is a projection of world point X W , (u,v) C2 is another feature point in an image from a second camera c2 that is a projection of the world point X W , R V2Ci is the rotation matrix of camera c i in vehicle coordinates v, i is the index of the cameras, t is the translation vector, dyn represents vehicle dynamic, and w is world coordinates.

7. The method according to claim 1 wherein estimating vehicle dynamics includes providing temporal tracking and smoothing.

8. The method according to claim 1 wherein the surround-view camera system includes four cameras, wherein a first camera is positioned at a front of the vehicle, a second camera is positioned at a back of the vehicle, a third camera is positioned at a left side of the vehicle and a fourth camera is positioned at a right side of the vehicle.

9. The method according to claim 8 wherein images from the first camera overlap with images from the third camera, images from the first camera overlap with images from the fourth camera, images from the second camera overlap with images from the third camera, and images from the second camera overlap with images from the fourth camera.

10. A method for estimating platform dynamics using images from cameras on a mobile platform, said method comprising:
providing digital images from the cameras to a processor;
identifying one or more overlap image areas for any two cameras;
identifying, using the processor, common feature points in the one or more overlap image areas;
determining, using the processor, that the identified common feature points in the overlap image areas are not at the same location in the overlap image areas;
estimating, using the processor, three-degree of freedom dynamic parameters from comparisons between the common feature points; and
estimating, using the processor, platform dynamics of one or more of pitch, roll and height variation using the dynamic parameters.

11. The method according to claim 10 wherein identifying common feature points includes identifying common feature points in the images from two cameras that are projections of the same point location in world coordinates.

12. The method according to claim 10 wherein identifying common feature points identifying a region of interest in the overlap image areas that includes the common feature points.

13. The method according to claim 10 wherein estimating platform dynamics includes solving dynamic equations to obtain an estimate of the pitch, roll and/or height variation of the platform based on a difference between the common feature points.

14. The method according to claim 10 wherein estimating platform dynamics includes determine a rotation matrix and a translation vector.

15. The method according to claim 10 wherein the plurality of cameras are part of a surround-view camera system.

16. The method according to claim 15 wherein the surround-view camera system includes four cameras, wherein a first camera is positioned at a front of the platform, a second camera is positioned at a back of the platform, a third camera is positioned at a left side of the platform and a fourth camera is positioned at a right side of the platform.

17. The method according to claim 16 wherein images from the first camera overlap with images from the third camera, images from the first camera overlap with images from the fourth camera, images from the second camera overlap with images from the third camera, and images from the second camera overlap with images from the fourth camera.

18. A dynamics system for estimating vehicle dynamics using images from cameras in a surround-view camera system on a vehicle, said system comprising:
means for providing digital images from the cameras to a processor;
means for identifying one or more overlap image areas for any two cameras in the surround-view camera system;
means for identifying, using the processor, common feature points in the one or more overlap image areas;
means for determining, using the processor, that the identified common feature points are not at the same location in the overlap image areas;
means for estimating, using the processor, three-degree of freedom vehicle dynamic parameters from comparisons between the common feature points; and
means for estimating, using the processor, vehicle dynamics of one or more of pitch, roll and height variation using the vehicle dynamic parameters.

19. The system according to claim 18 wherein the means for identifying common feature points identifies a region of interest in the overlap image areas that includes the common feature points.

20. The system according to claim 18 wherein the means for identifying common feature points identifies common feature points in the images from two cameras that are projections of the same point location in world coordinates.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Wang Jinsong
• Zhang Wende
• Litkouhi Bakhtiar B

Classifications

• G06T7/0042

  Physics · mapped topic

• G06T7/0071

  Physics · mapped topic

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06K9/3241

  Physics · mapped topic

• G06T7/2033 Primary

  Physics · mapped topic