Velocity estimation and angle offset correction in SAR images by performing image matching

Patent metadata

Field	Value
Publication number	US-12313733-B2
Application number	US-202217901792-A
Country	US
Kind code	B2
Filing date	Sep 1, 2022
Priority date	Sep 1, 2022
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

A method, system and vehicle that repetitively correct angle offsets in a synthetic aperture radar image of a vehicle while the vehicle is in motion by utilizing a radar system and a camera to determine accurate velocity of a measured object by matching angles of the object in the SAR image with angles of the object in the camera image, thereby reducing angle offsets of objects in the SAR image. The method includes obtaining an SAR image of another vehicle via a radar unit of the vehicle, obtaining a camera image of the other vehicle via a camera unit of the vehicle, determining an association between at least one object in the SAR image and a corresponding at least one object in the camera image, correcting a velocity estimation of the vehicle based on the determined association, and adjusting the SAR image based on the corrected velocity estimation.

First claim

What is claimed is:

1. A method for correcting a synthetic aperture radar (SAR) image captured by a first vehicle of a second vehicle, the method performed repetitively as the first vehicle is in motion and comprising:
obtaining an SAR image of the second vehicle via a radar unit of the first vehicle;
obtaining a camera image of the second vehicle via a camera unit of the first vehicle;
determining an association between objects in the SAR image and objects in the camera image;
correcting a velocity estimation of the first vehicle based on the determined association; and
adjusting the SAR image based on the corrected velocity estimation.

2. The method of claim 1 , wherein determining the association comprises identifying a pair of matching objects between the SAR image and the camera image.

3. The method of claim 1 , wherein adjusting the SAR image comprises performing coherent combining back-projection according to the corrected velocity.

4. The method of claim 2 , wherein correcting the velocity comprises finding an angle offset between the pair of matching objects.

5. The method of claim 2 , wherein identifying the matching pairs comprises:
extracting first deep neural network (DNN) features from a coarse object detection of each of the at least one object in the SAR image to obtain a radar descriptor vector for the corresponding each of the at least one object in the SAR image;
extracting second DNN features from an object detection of the corresponding each of the at least object in the camera image to obtain a camera descriptor vector for each of the at least one object in the camera image; and
performing bipartite matching between the camera descriptor vector for each of the at least one object in the SAR image and the camera descriptor vector for the corresponding each of the at least one object in the camera image.

6. The method of claim 5 , wherein identifying the matching pairs further comprises determining each of the at least one object in the SAR image and the corresponding at least one object in the camera image by adjusting DNN weights of the first DNN features and the second DNN features such that there is a minimum distance between each of the at least one object in the SAR image and the corresponding at least one object in the camera image.

7. The method of claim 1 , wherein obtaining the SAR image and obtaining the camera image are performed at the same time.

8. A system for repetitively correcting a synthetic aperture radar (SAR) image captured by a first vehicle of a second vehicle while the first vehicle is in motion, the system comprising:
a radar unit of the first vehicle obtaining an SAR image of the second vehicle;
a camera unit obtaining a camera image of the second vehicle;
a processor configured to:
determine an association between at least one object in the SAR image and a corresponding at least one object in the camera image;
correct a velocity estimation of the first vehicle based on the determined association; and
adjust the SAR image based on the corrected velocity estimation.

9. The system of claim 8 , wherein the processor is further configured to determine the association by identifying a pair of matching objects between the SAR image and the camera image.

10. The system of claim 8 , wherein the processor is further configured to adjust the SAR image by performing coherent combining back-projection according to the corrected velocity.

11. The system of claim 10 , wherein the processor is further configured to correct the velocity by finding an angle offset between the pair of matching objects.

12. The system of claim 9 , wherein the processor is configured to identify the matching pairs by:
extracting first deep neural network (DNN) features from a coarse object detection of each of the at least object in the SAR image to obtain a radar descriptor vector for the corresponding each of the at least one object in the SAR image;
extracting second DNN features from a coarse object detection of the corresponding each of the at least object in the camera image to obtain a camera descriptor vector for each of the at least one object in the camera image; and
performing bipartite matching between the camera descriptor vector for each of the at least one object in the SAR image and the camera descriptor vector for the corresponding each of the at least one object in the camera image.

13. The system of claim 12 , wherein the processor is further configured to identify the matching pairs by determining each of the at least one object in the SAR image and the corresponding at least one object in the camera image by adjusting DNN weights of the first DNN features and the second DNN features such that there is a minimum distance between each of the at least one object in the SAR image and the corresponding at least one object in the camera image.

14. The system of claim 8 , wherein the processor is further configured to obtain the SAR image and obtain the camera image at the same time.

15. A first vehicle for repetitively correcting a synthetic aperture radar (SAR) image while the first vehicle is in motion, the first vehicle comprising:
a vehicle body;
road wheels connected to the vehicle body;
a radar unit capturing synthetic aperture radar (SAR) images;
a camera capturing images; and
a control system, wherein the control system is configured to:
obtain an SAR image of a second vehicle via the radar unit;
obtain a camera image of the second vehicle via the camera unit;
determine an association between at least one object in the SAR image and a corresponding at least one object in the camera image;
correct a velocity estimation of the first vehicle based on the determined association; and
adjust the SAR image based on the corrected velocity estimation.

16. The vehicle of claim 15 , wherein the control system is further configured to determine the association by identifying a pair of matching objects between the SAR image and the camera image.

17. The vehicle of claim 15 , wherein the control system is further configured to adjust the SAR image by performing coherent combining back-projection according to the corrected velocity.

18. The system of claim 16 , wherein the control system is further configured to correct the velocity by finding an angle offset between the pair of matching objects.

19. The system of claim 16 , wherein the control system is further configured to identify the matching pairs by:
extracting first deep neural network (DNN) features from a coarse object detection of each of the at least one object in the SAR image to obtain a radar descriptor vector for the corresponding each of the at least one object in the SAR image;
extracting second DNN features from an object detection of the corresponding each of the at least object in the camera image to obtain a camera descriptor vector for each of the at least one object in the camera image; and
performing bipartite matching between the camera descriptor vector for each of the at least one object in the SAR image and the camera descriptor vector for the corresponding each of the at least one object in the camera image.

20. The vehicle of claim 15 , wherein the control system is further configured to obtain the SAR image and obtain the camera image at the same time.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Bialer Oded
• Levi Dan

Classifications

• G01S13/589

  measuring the velocity vector · CPC title

• G01S13/931 Primary

  of land vehicles · CPC title

• G01S7/417

  involving the use of neural networks · CPC title

• G01S13/9029 Primary

  specially adapted for moving target detection within a single SAR image or within multiple SAR images taken at the same time · CPC title

• G01S13/9017

  with time domain processing of the SAR signals in azimuth (G01S13/9005 takes precedence) · CPC title