Model driven optimization of annotator execution in question answering system

Patent metadata

Field	Value
Publication number	US-10169074-B2
Application number	US-201514857903-A
Country	US
Kind code	B2
Filing date	Sep 18, 2015
Priority date	Apr 14, 2014
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

Mechanisms are provided for scheduling execution of pre-execution operations of an annotator of a question and answer (QA) system pipeline. A model is used to represent a system of annotators of the QA system pipeline, where the model represents each annotator as a node having one or more performance parameters indicating a performance of an execution of an annotator corresponding to the node. For each annotator in a set of annotators of the system of annotators, an effective response time for the annotator is calculated based on the performance parameters. A pre-execution start interval for a first annotator based on an effective response time of a second annotator is calculated where execution of the first annotator is sequentially after execution of the second annotator. Execution of pre-execution operations associated with the first annotator is scheduled based on the calculated pre-execution start interval for the first annotator.

First claim

What is claimed is:

1. A method, in a data processing system comprising a processor and a memory, for scheduling execution of pre-execution operations of an annotator of a question and answer (QA) system pipeline, the method comprising:
using, by the data processing system, a model to represent a system of annotators of the QA system pipeline, wherein the model represents each annotator in the system of annotators as a node having one or more performance parameters for indicating a performance of an execution of an annotator corresponding to the node, wherein each annotator in the s stem of annotators is a program that takes a portion of unstructured input text, extracts structured information from the portion of the unstructured input text, and generates annotations or metadata that are attached by the annotator to a source of the unstructured input text, wherein, for each node in the model, the one or more performance parameters corresponding to the node comprise an arrival rate parameter and a service rate parameter of the annotator associated with the node, wherein the arrival rate parameter indicates a number of jobs arriving in the node per second, and wherein the service rate parameter indicates a number of jobs being serviced by the node per second;
determining, by the data processing system, for each annotator in a set of annotators of the system of annotators, an effective response time for the annotator based on the one or more performance parameters;
calculating, by the data processing system, a pre-execution start interval for a first annotator based on an effective response time of a second annotator, wherein execution of the first annotator is sequentially after execution of the second annotator; and
scheduling, by the data processing system, execution of pre-execution operations associated with the first annotator based on the calculated pre-execution start interval for the first annotator.

2. The method of claim 1 , wherein at least one node in the model represents a plurality of annotators comprising an aggregate annotator.

3. The method of claim 1 , wherein the system is a tandem open queuing network in which each node in the model is modeled as a queue in the tandem open queuing network.

4. The method of claim 1 , wherein each annotator of the QA system pipeline is defined, in a data structure, as a sub-system of the QA system pipeline and is designated as either having or not having one or more non-dependent pre-execution operations that may be executed with the results of the one or more non-dependent pre-execution operations being stored/cached for use by the annotator, and wherein scheduling execution of pre-execution operations associated with the first annotator is performed in response to the data structure indicating that the pre-execution operations associated with the first annotator are non-dependent pre-execution operations.

5. The method of claim 1 , wherein calculating the pre-execution start interval for the first annotator based on an effective response time of the second annotator comprises calculating the pre-execution start interval based on a difference of a current execution time and an effective response time of the second annotator.

6. The method of claim 1 , wherein the scheduling generates a scheduling data structure, and wherein the method further comprises:
receiving a job for processing by the QA system pipeline;
selecting a set of annotators in the system of annotators to execute the job, the set of annotators comprising the first annotator and the second annotator;
scheduling the set of annotators, including the pre-execution operation of the first annotator, based on the scheduling data structure; and
processing the job based on the scheduling of the set of annotators.

Assignees

• IBM

Inventors

• Allen Corville O
• Carter Bernadette A
• Ghosh Rahul

Classifications

• G06F9/48 Primary

  Program initiating; Program switching, e.g. by interrupt · CPC title

• G06N5/041 Primary

  Abduction · CPC title

• G06F16/335

  Filtering based on additional data, e.g. user or group profiles (filtering in web context G06F16/9535, G06F16/9536) · CPC title

• G06F40/169

  Annotation, e.g. comment data or footnotes · CPC title

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title