Vehicle object tracking

Patent metadata

Field	Value
Publication number	US-11030774-B2
Application number	US-201916357713-A
Country	US
Kind code	B2
Filing date	Mar 19, 2019
Priority date	Mar 19, 2019
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

A system, comprising a computer that includes a processor and a memory, the memory storing instructions executable by the processor to determine an object location prediction based on a video data stream, wherein the object location prediction is based on processing cropped TEDA data with a neural network. The processor can be further programmed to download the object location prediction to a vehicle based on a location of the vehicle.

First claim

We claim:

1. A method, comprising:
determining an object location prediction based on a video stream data, wherein the object location prediction is based on processing cropped typicality and eccentricity data analytics (TEDA) data with a neural network by determining a first eccentricity image based on a per pixel average and a per pixel variance over a moving window of k video frames;
cropping the TEDA data based on a three-channel output image including a grayscale image, a positive eccentricity image determined by selecting pixels from the first eccentricity image when the pixels are greater than a per-pixel mean, and a negative eccentricity image determined by selecting pixels from the first eccentricity image when the pixels are less than the per-pixel mean; and
providing the object location prediction to a vehicle based on a location of the vehicle.

2. The method of claim 1 , further comprising determining TEDA data by processing the video stream data to determine an eccentricity image based on a per pixel average and a per pixel variance over a moving window of k video frames, wherein k is a small number.

3. The method of claim 2 , further comprising determining TEDA data by determining a three-channel output image including a grayscale image, a positive eccentricity e + image determined by selecting pixels from the first eccentricity image when the squares of the absolute value of the pixels are greater than the squares of the absolute value of the per-pixel mean, and negative eccentricity e − image determined by selecting pixels from the first eccentricity image when the squares of the absolute value of the pixels are less than the squares of the absolute value of the per-pixel mean.

4. The method of claim 1 , further comprising cropping the TEDA data based on the object location prediction, wherein an initial object location prediction is determined based on processing a frame of video stream data with the neural network and determining a minimal enclosing rectangle.

5. The method of claim 4 , wherein the cropped TEDA data is processed with a convolutional neural network to determine the object location prediction.

6. The method of claim 5 , wherein a first object location prediction is concatenated with intermediate results, or, at subsequent iterations, an object location prediction output at a previous iteration and processed with a fully-connected neural network to determine the object location prediction.

7. The method of claim 1 , further comprising determining the object location prediction based on global coordinates.

8. The method of claim 1 , further comprising providing the object location prediction based on global coordinates corresponding to the location of the vehicle.

9. The method of claim 1 , wherein the video stream data is acquired by a stationary video camera included in a traffic infrastructure system that includes a computing device to communicate with the vehicle via a wireless network.

10. A system, comprising a processor; and
a memory, the memory including instructions to be executed by the processor to:
determine an object location prediction based on a video stream data, wherein the object location prediction is based on processing cropped typicality and eccentricity data analytics (TEDA) data with a neural network by determining a first eccentricity image based on a per pixel average and a per pixel variance over a moving window of k video frames;
crop the TEDA data based on a three-channel output image including a grayscale image, a positive eccentricity image determined by selecting pixels from the first eccentricity image when the pixels are greater than a per-pixel mean, and a negative eccentricity image determined by selecting pixels from the first eccentricity image when the pixels are less than the per-pixel mean; and
provide the object location prediction to a vehicle based on a location of the vehicle.

11. The system of claim 10 , wherein the instructions further include instructions to determine TEDA data by processing the video stream data to determine an eccentricity e image based on a per pixel average and a per pixel variance over a moving window of k video frames, wherein k is a small number.

12. The system of claim 11 , wherein the instructions further include instructions to determine TEDA data by determining a three-channel output image including a grayscale image, a positive eccentricity e + image determined by selecting pixels from the first eccentricity image when the squares of the absolute value of the pixels are greater than the squares of the absolute value of the per-pixel mean, and negative eccentricity e − image determined by selecting pixels from the first eccentricity image when the squares of the absolute value of the pixels are less than the squares of the absolute value of the per-pixel mean.

13. The system of claim 10 , wherein the instructions further include instructions to crop the TEDA data based on the object location prediction, wherein an initial object location prediction is determined based on processing a frame of video stream data with the neural network and determining a minimal enclosing rectangle.

14. The system of claim 13 , wherein the instructions further include instructions to process the cropped TEDA data with a convolutional neural network to determine the object location prediction.

15. The system of claim 14 , wherein the instructions further include instructions to concatenate a first object location prediction with intermediate results, or, at subsequent iterations, an object location prediction output at a previous iteration and processed with a fully-connected neural network to determine the object location prediction.

16. The system of claim 10 , wherein the instructions further include instructions to determine the object location prediction based on global coordinates.

17. The system of claim 16 , wherein the instructions further include instructions to provide the object location prediction based on global coordinates corresponding to the location of the vehicle.

18. The system of claim 17 , wherein the video stream data is acquired by a stationary video camera included in a traffic infrastructure system that communicates with the vehicle via a wireless network.

19. A system, comprising:
means for controlling vehicle steering, braking and powertrain; and
means for:
determining an object location prediction based on a video stream data, wherein the object location prediction is based on processing cropped typicality and eccentricity data analytics (TEDA) data with a neural network by determining a first eccentricity image based on a per pixel average and a per pixel variance over a moving window of k video frames;
cropping the TEDA data based on a three-channel output image including a grayscale image, a positive eccentricity image determined by selecting pixels from the first eccentricity image when the pixels are greater than a per-pixel mean, and a negative eccentricity image determined by selecting pixels from the first eccentricity image when the pixels are less than the per-pixel mean; and
providing the object location prediction to a vehicle based on a location of the vehicle and the means for controlling vehicle steering, braking and powertrain.

20. The system of claim 19 , further comprising determining TEDA data by means for processing the video data based to determine an eccentricity e image based on per pixel average and per pixel variance over a moving window of k video frames, wherein k is a small

Assignees

• Ford Global Tech Llc

Inventors

• Parchami Mostafa
• Puskorius Gintaras Vincent
• Palanisamy Dinesh

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V20/40 Primary

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• G06V2201/08

  Detecting or categorising vehicles · CPC title