Systems and methods for automatically creating an image processing pipeline

Patent metadata

Field	Value
Publication number	US-2021141663-A1
Application number	US-201916681658-A
Country	US
Kind code	A1
Filing date	Nov 12, 2019
Priority date	Nov 12, 2019
Publication date	May 13, 2021
Grant date	—

Abstract

This disclosure relates to automatic creation of an image processing pipeline that has a dependency on image processing domain experts. Given the image processing problem has multiple viable solutions, search space of creating a solution for a given goal in a given constrained infrastructure is large and, choosing an optimal image processing solution is an effort, time and intellect intensive endeavor. The present disclosure models image processing goals in abstract task categories and externalizes domain knowledge as an object model and a set of rules defined over it. Rules codify task template selection criteria and algorithm selection criteria for implementing a task. Recommendations are provided to choose suitable algorithm (s) for carrying out image processing tasks. Effects of algorithm execution are recorded for subsequent tasks recommendations. The stored state of execution and trace information enables rerouting the execution and knowledge upgrade. Deployable code is generated after a validated pipeline is created.

First claim

What is claimed is:

1 . A processor implemented method for automatically creating an image processing pipeline, the method comprising the steps of:
receiving, via one or more hardware processors, an input data comprising an input goal and one or more images to be processed for achieving the input goal, the input data being characterized by a plurality of input properties corresponding to the input goal and the one or more images; creating a context model, via the one or more hardware processors, in the form of an instance of a context meta model, wherein the context meta model is configured to capture context of a domain having a property goalDescription corresponding to the input goal, and wherein the context meta model is characterized by a plurality of types of contexts including (i) a DomainContext representing domain related information, (ii) an InputContext representing information pertaining to the one or more images and (iii) a ProcessingContext representing information pertaining to the one or more hardware processors; creating a potential image processing pipeline, via the one or more hardware processors, based on a solution meta model, wherein the solution meta model is characterized by one or more algorithms associated with the DomainContext for performing a plurality of tasks corresponding to the input goal, and wherein each of the plurality of tasks are associated with (i) a TaskType from a plurality of TaskTypes (ii) a TaskTemplate representing a reusable task sequence for the input data using a taskOrder and (iii) the one or more algorithms for performing an associated task; creating a rule model, via the one or more hardware processors based on a rule meta model, wherein the rule meta model is configured to capture knowledge pertaining to the DomainContext and wherein the rule model is characterized by a plurality of rules of types including (i) TemplateRule representing rules for selecting the TaskTemplate from a list of TaskTemplate, for each of the plurality of tasks, (ii) AlgoRule representing rules for selecting an algorithm from the one or more algorithms for performing an associated task in the TaskTemplate based on the context model and (iii) PropRule representing rules for updating properties of the one or more images based on execution of a selected algorithm; obtaining, via the one or more hardware processors, the taskOrder representing an order of execution of the plurality of tasks by performing execution of the TemplateRule for selection of the TaskTemplate pertaining to the goalDescription; executing, via the one or more hardware processors, one or more rules from the plurality of rules, comprised in AlgoRule, corresponding to each of the plurality of tasks based on the obtained order of execution of the plurality of tasks to obtain a recommended algorithm for each of the plurality of tasks; executing, via the one or more hardware processors, the recommended algorithm for each of the plurality of tasks; and validating, via the one or more hardware processors, the potential image processing pipeline to obtain a validated image processing pipeline corresponding to the input goal using a continually updated model based Knowledge Repository comprising the context meta model, the solution meta model, the rule meta model and algorithmic services corresponding to the one or more algorithms, by dynamically performing:
on execution of the recommended algorithm, validating an outcome associated thereof for each of the plurality of tasks for input and output consistency; and
on execution of the one or more rules, validating the one or more rules corresponding to each of the plurality of tasks based on dynamic updation of the InputContext of the context model.

2 . The processor implemented method of claim 1 , further comprising:
generating a deployable code based on the validated image processing pipeline; and continually updating the Knowledge Repository with (i) the validated image processing pipeline, (ii) the deployable code, (iii) the context model, (iv) the potential image processing pipeline, (v) the rule model, (vi) each state of execution as a stack of state entries, wherein each state entry is a tuple comprising the TaskType, state of the context model characterized by the input properties, and {algorithm name, <parameter name, parameter value>} and (vii) trace information corresponding to navigation from one state of execution to another via the validated image processing pipeline.

3 . The processor implemented method of claim 1 , wherein the step of execution of the recommended algorithm comprises providing an output of a previous algorithm as a default value for one or more parameters as an input while executing a subsequent algorithm.

4 . The processor implemented method of claim 2 , wherein the step of validating the potential image processing pipeline comprises re-routing of the execution to a previous state of execution.

5 . The processor implemented method of claim 1 , wherein the step of validating an outcome associated with the recommended algorithm comprises modifying the taskOrder based on the outcome associated with each of the recommended algorithm for each of the plurality of tasks.

6 . The processor implemented method of claim 1 , wherein the step of executing the one or more rules is preceded by performing fuzzy transformation of numerical values associated with the input data to qualitative ranges.

7 . The processor implemented method of claim 1 , wherein the step of validating the one or more rules comprises executing the PropRule to cascade effects of changes in the context model.

8 . The processor implemented method of claim 1 , wherein (i) the domain related information comprises business domain and domain objects details; (ii) the information pertaining to the one or more images comprises image spectral and spatial characteristics, image contrast level, edge density, image format, noise information; band related information including directivity, entropy, linearity, periodicity, size and length; and sensor related information including azimuth angle, band width, interval at which the one or more images are taken, look angle, spatial and temporal resolution and zenith angle; and (iii) the information pertaining to the one or more hardware processors comprises processing related constraints including time constraint, batch or parallel processing and processing window size.

9 . The processor implemented method of claim 1 , wherein the plurality of TaskTypes include Transform, Prepare, Focus, Perceive and Quantify, wherein Transform and Prepare are configured to preprocess the one or more images to obtain transformed one or more images for further processing, Focus is configured to highlight regions of interest (ROIs) in the transformed one or more images and Perceive and Quantify are configured to interpret the ROIs in the one or more images to generate a required output for achieving the input goal, the ROIs being comprised in the plurality of input properties.

10 . The processor implemented method of claim 1 , wherein the plurality of rules are expressed using properties and values from the context model using RuleExpression.

11 . The processor implemented method of claim 9 , wherein the step of obtaining an order of execution of the plurality of tasks comprises using a default task type sequence of Transform, Prepare, Perceive, Focus, and Quantify if execution of the TemplateRule does not provide a TaskTemplate.

12 . A system for automatically creating an image processing pipeline, the system comprising one or more data storage devices operatively coupled to one or more hardware processors and configured to

Assignees

• Tata Consultancy Services Ltd

Inventors

• Deshpande Shailesh
• Rajbhoj Asha
• Gubbi Lakshminarasimha Jayavardhana Rama
• Kulkarni Vinay
• Purushothaman Balamuralidhar

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06N5/025 Primary

  Extracting rules from data · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N5/04

  Inference or reasoning models · CPC title