Automated classification and indexing of events using machine learning

Patent metadata

Field	Value
Publication number	US-12094493-B2
Application number	US-202117328960-A
Country	US
Kind code	B2
Filing date	May 24, 2021
Priority date	May 24, 2021
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

Described herein are techniques that may be used to automatically identify and index events within a media content file. Such techniques may comprise receiving, from at least one recording device, a media content, receiving sensor data determined to correspond to the media content, determine a context associated with the at least one recording device based on the sensor data, identifying, based on one or more data patterns detected within the sensor data and based on the contextual data, at least one event, generating an index corresponding to the identified event, and storing an indication of the generated index in association with the media content.

First claim

What is claimed is:

1. A method comprising:
receiving, from at least one recording device assigned to a person, a media content;
receiving sensor data determined to correspond to the media content;
receiving contextual data that are associated with the media content from a third-party service provider, the contextual data being generated by the third-party service provider based on additional sensor data that is received by an onboard computer of a vehicle assigned to the person and provided by the onboard computer to the third-party service provider;
identifying one or more data patterns of the sensor data and the contextual data, wherein the data patterns are based on one or more of a movement of the recording device, an audio cue of one or more of the sensor data and the contextual data, or a particular object depicted in the sensor data and the contextual data;
correlating the data patterns to at least one event included in the media content;
generating an index corresponding to an identified event of the at least one event, the index comprising at least a begin time for the identified event;
appending the index to the media content, wherein the index is appended to the media content via an event track that is separate from an audio track and video track of the media content; and
storing the indexed media content.

2. The method of claim 1 , wherein the media content is received from the at least one recording device in substantial real-time as streaming data.

3. The method of claim 1 , wherein the media content is received from the at least one recording device as an upload after the recording device has finished recording.

4. The method of claim 1 , wherein the media content comprises at least one of video data or audio data.

5. The method of claim 1 , wherein the sensor data comprises data obtained from at least one of a gyroscope, accelerometer, or compass.

6. The method of claim 1 , wherein an indication of the generated index is stored in a database table that is mapped to the media content.

7. The method of claim 1 , further comprising upon determining that the identified event corresponds to a category of event that is flagged for review, providing a notification to at least one user that includes the generated index.

8. A computing device comprising:
a processor;
a memory storing instructions that, when executed with the processor, cause the computing device to, at least:
receive, from at least one recording device, a media content;
receive sensor data determined to correspond to the media content;
determine a context associated with the at least one recording device based on the sensor data;
receive additional contextual data from an external device that is in proximity of a recording device of the at least one recording device and in direct short-range wireless communication with the recording device;
identify one or more data patterns of the sensor data and the contextual data, wherein the data patterns are based on one or more of the sensor data and the contextual data, or a particular object depicted in the sensor data and the contextual data;
correlate the data patterns to at least one event included in the media content;
generate an index corresponding to an identified event of the at least one event;
append the index to the media content, wherein the index is appended to the media content via an event track that is separate from an audio track and video track of the media content, the index for accessing media data in the audio track and the video track that correlates to the identified event; and
store the indexed media content.

9. The computing device of claim 8 , wherein the context is determined using a first trained machine learning model and the identified event is identified using a second trained machine learning model.

10. The computing device of claim 8 , wherein the additional contextual data includes vehicle data received from an onboard computer of a vehicle.

11. The computing device of claim 10 , wherein the additional contextual data includes data received from a firearm that indicates the firearm has been discharged.

12. The computing device of claim 8 , wherein the generated index comprises at least one of an event identifier and a timestamp.

13. The computing device of claim 12 , wherein the timestamp represents a single point in time.

14. The computing device of claim 12 , wherein the timestamp represents a range of times.

15. The computing device of claim 8 , wherein the sensor data is received from the at least one recording device.

16. The computing device of claim 8 , wherein an indication of the generated index is stored within an event data track of the media content.

17. The computing device of claim 8 , wherein selection of the generated index using a media player causes the media player to play the media content from a particular point on a timeline associated with the media content, the particular point in time corresponding to an occurrence of the event.

18. A non-transitory computer-readable media collectively storing computer-executable instructions that upon execution cause one or more computing devices to collectively perform acts comprising:
receiving, from at least one recording device assigned to a person, a media content;
receiving sensor data determined to correspond to the media content;
receiving contextual data that are associated with the media content from a third-party service provider, the contextual data being generated by the third-party service provider based on additional sensor data that is received by an onboard computer of a vehicle assigned to the person and provided by the onboard computer to the third-party service provider;
identifying one or more data patterns of the sensor data and the contextual data, wherein the data patterns are based on one or more of a movement of the recording device, an audio cue of one or more of the sensor data and the contextual data, or a particular object depicted in the sensor data and the contextual data;
correlating the data patterns to at least one event included in the media content;
generating an index corresponding to an identified event of the at least one event, the index comprising at least a begin time for the identified event;
appending the index to the media content, wherein the index is appended to the media content via an event track that is separate from an audio track and video track of the media content; and
storing the indexed media content.

19. The non-transitory computer-readable media of claim 18 , wherein the media content is received from the at least one recording device as streaming data.

20. The non-transitory computer-readable media of claim 19 , wherein the sensor data is received from the at least one recording device.

Assignees

• Getac Technology Corp
• Whp Workflow Solutions Inc

Inventors

• Guzik Thomas
• Adeel Muhammad

Classifications

• G06V20/44

  Event detection · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06F16/41 Primary

  Indexing; Data structures therefor; Storage structures · CPC title

• G06N20/20

  Ensemble learning · CPC title

• G11B27/34

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title