Analysis of video content

Patent metadata

Field	Value
Publication number	US-10534965-B2
Application number	US-201815926745-A
Country	US
Kind code	B2
Filing date	Mar 20, 2018
Priority date	Nov 22, 2017
Publication date	Jan 14, 2020
Grant date	Jan 14, 2020

Abstract

Techniques for analyzing stored video upon a request are described. For example, a method of receiving a first application programming interface (API) request to analyze a stored video, the API request to include a location of the stored video and at least one analysis action to perform on the stored video; accessing the location of the stored video to retrieve the stored video; segmenting the accessed video into chunks; processing each chunk with a chunk processor to perform the at least one analysis action, each chunk processor to utilize at least one machine learning model in performing the at least one analysis action; joining the results of the processing of each chunk to generate a final result; storing the final result; and providing the final result to a requestor in response to a second API request is described.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving a first application programming interface (API) request to analyze a stored video, the API request to include a location of the stored video and at least one analysis action to perform on the stored video;
placing the request into a queue;
polling the queue to retrieve the request;
accessing the location of the stored video to retrieve the stored video;
segmenting the accessed video into chunks;
processing each chunk with a chunk processor to perform the at least one analysis action, each chunk processor to utilize at least one machine learning model in performing the at least one analysis action;
joining the results of the processing of each chunk to generate a final result;
storing the final result; and
providing the final result to a requestor in response to a second API request.

2. The computer-implemented method of claim 1 , wherein first API request is one of a start content moderation request to perform an analysis of content of the stored video, a start face detection request to perform face detection in the stored video, a start label detection request to perform label detection in the stored video, a start person tracking request to perform person tracking in the stored video, and a start celebrity recognition request to perform celebrity detection in the stored video.

3. The computer-implemented method of claim 1 , wherein the chunk processor includes a chunk decoder to generate chunk frames that are passed to at least one machine learning algorithm to perform the at least one analysis.

4. A computer-implemented method comprising:
receiving a first application programming interface (API) request to analyze a stored video, the API request to include a location of the stored video and at least one analysis action to perform on the stored video;
accessing the location of the stored video to retrieve the stored video;
segmenting the accessed video into chunks;
processing each chunk with a chunk processor to perform the at least one analysis action, each chunk processor to utilize at least one machine learning model in performing the at least one analysis action;
joining the results of the processing of each chunk to generate a final result;
storing the final result; and
providing the final result to a requestor in response to a second API request.

5. The computer-implemented method of claim 4 , wherein first API request is one of a start content moderation request to perform an analysis of content of the stored video, a start face detection request to perform face detection in the stored video, a start label detection request to perform label detection in the stored video, a start person tracking request to perform person tracking in the stored video, and a start celebrity recognition request to perform celebrity detection in the stored video.

6. The computer-implemented method of claim 4 , wherein the chunk processor includes a chunk decoder to generate chunk frames that are passed to at least one machine learning algorithm to perform the at least one analysis.

7. The computer-implemented method of claim 6 , wherein the least one machine learning algorithm of the chunk processor is a face detection algorithm.

8. The computer-implemented method of claim 6 , wherein the least one machine learning algorithm of the chunk processor is a label detection algorithm.

9. The computer-implemented method of claim 4 , wherein the joining of the results of the processing of each chunk to generate a final result is performed by an aggregator and the final result includes at least one of a per frame person bounding box, face bounding box, and a face match.

10. The computer-implemented method of claim 4 , wherein the first API request is received by a front end of a video analysis service.

11. The computer-implemented method of claim 4 , wherein the stored video is generated by capturing and indexing streaming video.

12. The computer-implemented method of claim 4 , wherein the stored video is encrypted and is protected from unauthorized access.

13. The computer-implemented method of claim 4 , further comprising:
notifying a requestor that the final result is available.

14. A system comprising:
an end user device to send a first request for analysis of stored video;
a web services provider to:
receive the request to analyze a stored video, the first request to include a location of the stored video and at least one analysis action to perform on the stored video;
access the location of the stored video to retrieve the stored video;
segment the accessed video into chunks;
process each chunk with a chunk processor to perform the at least one analysis action, each chunk processor to utilize at least one machine learning model in performing the at least one analysis action;
join the results of the processing of each chunk to generate a final result;
store the final result; and
provide the final result to a requestor in response to a second API request.

15. The system of claim 14 , wherein first request is one of a start content moderation request to perform an analysis of content of the stored video, a start face detection request to perform face detection in the stored video, a start label detection request to perform label detection in the stored video, a start person tracking request to perform person tracking in the stored video, and a start celebrity recognition request to perform celebrity detection in the stored video.

16. The system of claim 14 , wherein the chunk processor is to include a chunk decoder to generate chunk frames that are passed to at least one machine learning algorithm to perform the at least one analysis.

17. The system of claim 16 , wherein the least one machine learning algorithm of the chunk processor is a face detection algorithm.

18. The system of claim 16 , wherein the least one machine learning algorithm of the chunk processor is a label detection algorithm.

19. The system of claim 14 , wherein the joining of the results of the processing of each chunk to generate a final result is performed by an aggregator and the final result includes at least one of a per frame person bounding box, face bounding box, and a face match.

20. The system of claim 14 , wherein the first request is received by a front end of the web services provider.

Assignees

• Amazon Tech Inc

Inventors

• Singhal Nitin
• Bhadauria Vivek
• Das Ranju
• Ghare Gaurav D
• Goldenberg Roman
• Gould Stephen
• Han Kuang
• Hedley Jonathan Andrew
• Jeyabalan Gowtham
• Manohar Vasant
• Olgiati Andrea
• Stefani Stefano
• Tighe Joseph Patrick
• Udayakumar Praveen Kumar
• Zheng Renjun

Classifications

• G06K9/00765

  Physics · mapped topic

• G06K9/00744 Primary

  Physics · mapped topic

• G06K2009/00738

  Physics · mapped topic

• G06K9/00718

  Physics · mapped topic

• G06K9/00228

  Physics · mapped topic