Object detection

Patent metadata

Field	Value
Publication number	US-12046047-B2
Application number	US-202117544050-A
Country	US
Kind code	B2
Filing date	Dec 7, 2021
Priority date	Dec 7, 2021
Publication date	Jul 23, 2024
Grant date	Jul 23, 2024

Abstract

A segmentation mask can be determined that includes at least one moving object in a plurality of images based on determining eccentricity for each pixel location in the plurality of images. A first image included in the plurality of images can be segmented by applying the segmentation mask to the image. The segmented first image can be transformed to a compressed dense matrix which includes pixel values for non-zero portions of the segmented first image. The compressed dense matrix can be input to a sparse convolutional neural network trained to detect objects. A detected object corresponding to the at least one moving object included in the first image can be output from the sparse convolutional neural network.

First claim

The invention claimed is:

1. A computer, comprising:
a processor; and
a memory, the memory including instructions executable by the processor to:
determine a segmentation mask that includes at least one moving object in a plurality of images based on determining eccentricity for each pixel location in the plurality of images;
segment a first image included in the plurality of images by applying the segmentation mask to the image;
transform the segmented first image to a compressed dense matrix which includes pixel values for non-zero portions of the segmented first image;
input the compressed dense matrix to a sparse convolutional neural network trained to detect objects; and
output a detected object corresponding to the at least one moving object included in the first image from the sparse convolutional neural network.

2. The computer of claim 1 , the instructions including further instructions to operate a vehicle by determining a vehicle path based on the detected object.

3. The computer of claim 2 , wherein operating the vehicle on the vehicle path includes controlling one or more of vehicle powertrain, vehicle steering, and vehicle brakes.

4. The computer of claim 1 , wherein the plurality of images corresponds to images acquired at a plurality of time steps by a camera viewing a traffic scene.

5. The computer of claim 1 , wherein the at least one moving object includes one or more of a vehicle and a pedestrian.

6. The computer of claim 1 , wherein the pixel locations correspond to pixel addresses in a rectangular array of pixels included in each of the plurality of images.

7. The computer of claim 1 , wherein the eccentricity is determined based on determining a mean pixel value for each pixel location and a variance for each pixel location.

8. The computer of claim 1 , wherein pixels of the segmented first image are set to the eccentricity when the eccentricity is greater than a user-determined threshold and zero when the eccentricity is less than a user-determined threshold.

9. The computer of claim 1 , wherein applying the segmentation mask to the first image includes determining a logical AND between each pixel of the segmentation mask and a corresponding pixel of the first image.

10. The computer of claim 1 , wherein the compressed dense matrix includes an x, y pixel address and a pixel value for each non-zero pixel included in the segmented first image.

11. The computer of claim 10 , wherein the sparse convolutional neural network inputs the compressed dense matrix and outputs an array which includes an x, y pixel address of a bounding box and an object label corresponding to an object class.

12. The computer of claim 1 , wherein the sparse convolutional neural network includes a plurality of convolutional layers and a plurality of max pooling layers.

13. The computer of claim 1 , wherein the sparse convolutional neural network is trained to detect objects based on a training dataset that includes a plurality of sets of images and ground truth data corresponding to moving objects included in the pluralities of sets of images, respectively.

14. The computer of claim 13 , wherein the ground truth data includes object labels and bounding boxes corresponding to object locations for the moving objects included in the plurality of sets of images.

15. A method, comprising:
determining a segmentation mask that includes at least one moving object in a plurality of images based on determining eccentricity for each pixel location in the plurality of images;
segmenting a first image included in the plurality of images by applying the segmentation mask to the image;
transforming the segmented first image to a compressed dense matrix which includes pixel values for non-zero portions of the segmented first image;
inputting the compressed dense matrix to a sparse convolutional neural network trained to detect objects; and
outputting a detected object corresponding to the at least one moving object included in the first image from the sparse convolutional neural network.

16. The method of claim 15 , further comprising operating a vehicle by determining a vehicle path based on the detected object.

17. The method of claim 16 , wherein operating the vehicle on the vehicle path includes controlling one or more of vehicle powertrain, vehicle steering, and vehicle brakes.

18. The method of claim 15 , wherein the plurality of images corresponds to images acquired at a plurality of time steps by a camera viewing a traffic scene.

19. The method of claim 15 , wherein the at least one moving object includes one or more of a vehicle and a pedestrian.

20. The method of claim 15 , wherein the pixel locations correspond to pixel addresses in a rectangular array of pixels included in each of the plurality of images.

Assignees

• Ford Global Tech Llc

Inventors

• Parchami Mostafa
• Li Xiaomin
• Corona Enrique
• Singh Kunjan

Classifications

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G06T7/10

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title

• B60W30/09

  Taking automatic action to avoid collision, e.g. braking and steering · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06V10/82

  using neural networks · CPC title