System and method for contexually interpreting image sequences

Patent metadata

Field	Value
Publication number	US-9959470-B2
Application number	US-201615142345-A
Country	US
Kind code	B2
Filing date	Apr 29, 2016
Priority date	Jun 28, 2011
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

A system and method for contextually interpreting image sequences are provided. The method comprises receiving video from one or more video sources, and generating one or more questions associated with one or more portions of the video based on at least one user-defined objective. The method further comprises sending the one or more portions of the video and the one or more questions to one or more assistants, receiving one or more answers to the one or more questions from the one or more assistants, and determining a contextual interpretation of the video based on the one or more answers and the video.

First claim

What is claimed is:

1. A system comprising:
a processor;
a computer readable hardware storage device; and
program instructions recorded on the computer readable hardware storage device for execution by the processor, the program instructions comprising:
program instructions to receive video from one or more video sources;
program instructions to determine that one or more Application Video Packets (AVPs) of the video require user assistance for analyzing based on user-defined criteria;
program instructions to determine one or more questions associated with the one or more AVPs of the video that require user assistance based on at least one of the user-defined criteria and predetermined criteria;
program instructions to append each of the one or more questions to respective ones of the one or more AVPs;
program instructions to send the one or more AVPs and the one or more questions to one or more processor stations;
program instructions to receive one or more answers to the one or more questions from the one or more processor stations;
program instructions to collate the one or more answers;
program instructions to determine a contextual interpretation of the video based on the one or more answers, the predetermined criteria, and the video;
program instructions to parse the video based on the user-defined criteria;
program instructions to determine whether the parsing of the video requires user assistance based on the user-defined criteria; and
program instructions to determine the one or more AVPs based on the user-defined criteria when the parsing of the video requires user assistance.

2. The system of claim 1 , further comprising program instructions to change an order of the one or more AVPs before the sending of the one or more AVPs such that the one or more processor stations receiving the one or more AVPs do not receive consecutive AVPs in a same video sequence.

3. The system of claim 1 , further comprising program instructions to send the one or more AVPs and the one or more questions to the one or more processor stations which are operated by one or more users specifically-trained to answer the one or more questions based on the one or more AVPs.

4. The system of claim 1 , further comprising program instructions to identify one or more portions of the video for which the system requires assistance in the determining the contextual interpretation based on at least one user-defined objective for the contextual interpretation.

5. The system of claim 4 , wherein the at least one user-defined objective for the contextual interpretation comprises an answer that quantifies one or more objects in the video.

6. A system comprising:
a processor;
a computer readable hardware storage device; and
program instructions recorded on the computer readable hardware storage device for execution by the processor, the program instructions comprising:
program instructions to receive video from one or more video sources;
program instructions to determine that one or more Application Video Packets (AVPs) of the video require user assistance for analyzing based on user-defined criteria;
program instructions to determine one or more questions associated with the one or more AVPs of the video that require user assistance based on at least one of the user-defined criteria and predetermined criteria;
program instructions to append each of the one or more questions to respective ones of the one or more AVPs;
program instructions to send the one or more AVPs and the one or more questions to one or more processor stations;
program instructions to receive one or more answers to the one or more questions from the one or more processor stations;
program instructions to collate the one or more answers;
program instructions to determine a contextual interpretation of the video based on the one or more answers, the predetermined criteria, and the video; and
program instructions to identify one or more portions of the video for which the system requires assistance in the determining the contextual interpretation based on at least one user-defined objective for the contextual interpretation,
wherein the at least one user-defined objective comprising identifying one or more items possessed by a person in the video based on one or more criterion defined by the user.

7. A system comprising:
a processor;
a computer readable hardware storage device; and
program instructions recorded on the computer readable hardware storage device for execution by the processor, the program instructions comprising:
program instructions to receive video from one or more video sources;
program instructions to determine that one or more Application Video Packets (AVPs) of the video require user assistance for analyzing based on user-defined criteria;
program instructions to determine one or more questions associated with the one or more AVPs of the video that require user assistance based on at least one of the user-defined criteria and predetermined criteria;
program instructions to append each of the one or more questions to respective ones of the one or more AVPs;
program instructions to send the one or more AVPs and the one or more questions to one or more processor stations;
program instructions to receive one or more answers to the one or more questions from the one or more processor stations;
program instructions to collate the one or more answers;
program instructions to determine a contextual interpretation of the video based on the one or more answers, the predetermined criteria, and the video; and
program instructions to identify one or more portions of the video for which the system requires assistance in the determining the contextual interpretation based on at least one user-defined objective for the contextual interpretation,
wherein the at least one user-defined objective comprises identifying a person having difficulty breathing in the video based on one or more criterion defined by the user.

Assignees

• IBM

Inventors

• Hariharan Rajaraman
• Ramanathan Sri
• Subbian Karthik
• Trevathan Matthew B

Classifications

• G06K2009/00738

  Physics · mapped topic

• G06F17/3087

  Physics · mapped topic

• H04H60/48 Primary

  for recognising items expressed in broadcast information · CPC title

• H04N21/4758

  for providing answers, e.g. voting · CPC title

• G06F8/43

  Checking; Contextual analysis · CPC title