System and method for vehicle wheel detection

Patent metadata

Field	Value
Publication number	US-11967140-B2
Application number	US-202217983129-A
Country	US
Kind code	B2
Filing date	Nov 8, 2022
Priority date	Mar 10, 2017
Publication date	Apr 23, 2024
Grant date	Apr 23, 2024

Abstract

A system and method for vehicle wheel detection is disclosed. A particular embodiment can be configured to: receive training image data from a training image data collection system; obtain ground truth data corresponding to the training image data; perform a training phase to train one or more classifiers for processing images of the training image data to detect vehicle wheel objects in the images of the training image data; receive operational image data from an image data collection system associated with an autonomous vehicle; and perform an operational phase including applying the trained one or more classifiers to extract vehicle wheel objects from the operational image data and produce vehicle wheel object data.

First claim

What is claimed is:

1. A system comprising:
a data processor;
a memory for storing a detection system, executable by the data processor; and
an image data collection system associated with an autonomous vehicle, the image data collection system being in data communication with the data processor, the detection system being configured to:
receive, by use of the data processor, image data from the image data collection system;
conform, by use of the data processor, the image data to a predetermined image size;
extract, by use of the data processor, a vehicle wheel object of a vehicle other than the autonomous vehicle from the image data using at least one trained classifier, the at least one classifier being trained with ground truth data and training image data from a training image data collection system;
produce, by use of the data processor, vehicle wheel object data related to a wheel of the vehicle from the extracted vehicle wheel object, the vehicle wheel object data comprising vehicle wheel contour data corresponding to a contour surrounding the wheel of the vehicle, the vehicle wheel object data further comprising a predicted label map visualizing the vehicle wheel object data; and
infer, by use of the data processor, a pose, location, intention, and trajectory of the vehicle from which the vehicle wheel object is extracted based on the vehicle wheel object data and the predicted label map.

2. The system of claim 1 wherein the ground truth data is obtained from a manual image annotation or labeling process.

3. The system of claim 1 wherein the predicted label map includes a blended visualization of a raw image combined with the ground truth data.

4. The system of claim 1 being further configured to generate the ground truth data by filling in interior regions defined by contours of the extracted vehicle wheel objects.

5. The system of claim 1 being further configured to use a fully convolutional neural network (FCN) as a machine learning model.

6. The system of claim 1 being further configured to use a fully convolutional neural network (FCN) as a machine learning model with semantic segmentation.

7. The system of claim 1 being configured to generate object-level contour detections for each extracted vehicle wheel object of the image data.

8. A method comprising:
receiving, by use of a data processor, image data from an image data collection system;
conforming, by use of the data processor, the image data to a predetermined image size;
extracting, by use of the data processor, a vehicle wheel object of a vehicle other than the autonomous vehicle from the image data using at least one trained classifier, the at least one classifier being trained with ground truth data and training image data from a training image data collection system;
producing, by use of the data processor, vehicle wheel object data related to a wheel of the vehicle from the extracted vehicle wheel object, the vehicle wheel object data comprising vehicle wheel contour data corresponding to a contour surrounding the wheel of the vehicle, the vehicle wheel object data further comprising a predicted label map visualizing the vehicle wheel object data; and
inferring, by use of the data processor, a pose, location, intention, and trajectory of the vehicle from which the vehicle wheel object is extracted based on the vehicle wheel object data and the predicted label map.

9. The method of claim 8 including training the at least one classifier with ground truth data and training image data.

10. The method of claim 8 wherein the predicted label map includes a blended visualization of a raw image combined with the ground truth data.

11. The method of claim 8 including generating the ground truth data by filling in interior regions defined by contours of the extracted vehicle wheel objects.

12. The method of claim 8 including using a fully convolutional neural network (FCN) as a machine learning model.

13. The method of claim 8 including using a fully convolutional neural network (FCN) as a machine learning model with semantic segmentation using dense upsampling convolution (DUC).

14. The method of claim 8 including generating object-level contour detections for each extracted vehicle wheel object of the image data.

15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive image data from a image data collection system;
conform the image data to a predetermined image size;
extract a vehicle wheel object of a vehicle other than the autonomous vehicle from the image data using at least one trained classifier, the at least one classifier being trained with ground truth data and training image data from a training image data collection system;
produce vehicle wheel object data related to a wheel of the vehicle from the extracted vehicle wheel object, the vehicle wheel object data comprising vehicle wheel contour data corresponding to a contour surrounding the wheel of the vehicle, the vehicle wheel object data further comprising a predicted label map visualizing the vehicle wheel object data; and
infer a pose, location, intention, and trajectory of the vehicle from which the vehicle wheel object is extracted based on the vehicle wheel object data and the predicted label map.

16. The non-transitory machine-useable storage medium of claim 15 wherein the instructions being further configured to train the at least one classifier with ground truth data and training image data.

17. The non-transitory machine-useable storage medium of claim 15 wherein the instructions being further configured to generate the predicted label map with a blended visualization of a raw image combined with the ground truth data.

18. The non-transitory machine-useable storage medium of claim 15 wherein the instructions being further configured to generate the ground truth data by filling in interior regions defined by contours of the extracted vehicle wheel objects.

19. The non-transitory machine-useable storage medium of claim 15 wherein the instructions being further configured to use a fully convolutional neural network (FCN) as a machine learning model.

20. The non-transitory machine-useable storage medium of claim 15 wherein the instructions being configured to generate object-level contour detections for each extracted vehicle wheel object of the operational image data.

Assignees

• Tusimple Inc

Inventors

• Wang Panqu
• Chen Pengfei

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/24133

  Distances to prototypes · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title