Multiple exposure event determination

Patent metadata

Field	Value
Publication number	US-11840239-B2
Application number	US-202017122508-A
Country	US
Kind code	B2
Filing date	Dec 15, 2020
Priority date	Sep 29, 2017
Publication date	Dec 12, 2023
Grant date	Dec 12, 2023

Abstract

Systems, devices and methods provide, implement, and use vision-based methods of sequence inference for a device affixed to a vehicle.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, by a computer, video data from a camera on a vehicle;
detecting, by the computer, at least one object in a plurality of frames of the video data;
generating, by the computer, a bounding box corresponding to each of the at least one object in each of the plurality of frames in which the at least one object appears;
determining, by the computer, an inference for each bounding box in each of the plurality of frames;
superpositioning, by the computer, each bounding box with each inference in each of the plurality of frames so that the inference for each bounding box in each of the plurality of frames appears; and
identifying, by the computer, a sequence based on superpositioning each bounding box with each inference in each of the plurality of frames, where a traffic event is determined from the sequence of inferences.

2. The computer-implemented method according to claim 1 , further comprising generating, by the computer, a data vector based on superpositioning the inference for each bounding box in each of the plurality of frames.

3. The computer-implemented method according to claim 2 , wherein the data vector has a predetermined size.

4. The computer-implemented method according to claim 2 , wherein identifying the sequence comprises inputting, by the computer, the data vector into a neural network.

5. The computer-implemented method according to claim 2 , further comprising detecting, by the computer, an event based on the sequence.

6. The computer-implemented method according to claim 2 , further comprising determining, by the computer, a likelihood that an event has occurred based on the sequence.

7. The computer-implemented method according to claim 1 , further comprising identifying, by the computer, a pattern based on the sequence.

8. The computer-implemented method according to claim 7 , wherein the pattern indicates that the vehicle passed another vehicle.

9. The computer-implemented method according to claim 7 , wherein the pattern indicates that traffic is moving at nearly the same speed as the vehicle.

10. The computer-implemented method according to claim 1 , wherein a first inference for a first bounding box in a first frame of the plurality of frames comprises a first relative position between the vehicle and a second vehicle, and wherein a second inference for a second bounding box in a second frame of the plurality of frames comprises a second relative position between the vehicle and the second vehicle.

11. The computer-implemented method according to claim 1 , wherein generating the bounding box comprises identifying, by the computer, a corner coordinate, height, and width for the bounding box surrounding the object.

12. The computer-implemented method according to claim 1 , further comprising annotating, by the computer, the object corresponding to each bounding box.

13. The computer-implemented method according to claim 12 , wherein annotating comprises detecting, by the computer, a class of the object using a neural network.

14. The computer-implemented method according to claim 1 , wherein a first inference for a first bounding box comprises a relative position between a traffic control device and the vehicle.

15. The computer-implemented method according to claim 14 , wherein the traffic control device is a traffic light, and wherein a second inference of a first bounding box comprises an estimate of an illuminated color of the traffic light.

16. A system comprising:
a non-transitory computer-readable medium storing instructions; and
a processor configured to execute the instructions to:
receive video data from a camera on a vehicle;
detect at least one object in a plurality of frames of the video data;
generate a bounding box corresponding to each of the at least one object in each of the plurality of frames in which the at least one object appears;
determine an inference for each bounding box in each of the plurality of frames;
superposition each bounding box with each inference in each of the plurality of frames so that the inference for each bounding box in each of the plurality of frames appears; and
identify a sequence based on superpositioning each bounding box with each inference in each of the plurality of frames, where a traffic event is determined from the sequence of inferences.

17. The system according to claim 16 , wherein the processor is further executed to execute instructions to generate a data vector based on superpositioning the inference for each bounding box in each of the plurality of frames.

18. The system according to claim 16 , wherein the processor is further executed to execute instructions to input the data vector into a neural network.

19. The system according to claim 16 , wherein the processor is further executed to execute instructions to determine a likelihood that an event has occurred based on the sequence.

20. The system according to claim 16 , wherein the processor is further executed to execute instructions to identify a pattern based on the sequence.

Assignees

• Netradyne Inc

Inventors

• Annapureddy Venkata Sreekanta Reddy
• Campos Michael
• Yedla Arvind
• Julian David Jonathan

Classifications

• B60W40/04 Primary

  Traffic conditions · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06T7/248

  involving reference images or patches · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06V20/44

  Event detection · CPC title