Distributive hierarchical model for object recognition in video

Patent metadata

Field	Value
Publication number	US-9536177-B2
Application number	US-201414557262-A
Country	US
Kind code	B2
Filing date	Dec 1, 2014
Priority date	Dec 1, 2013
Publication date	Jan 3, 2017
Grant date	Jan 3, 2017

Abstract

Various examples are provided for object recognition in video. In one example, among others, a system includes processing circuitry including a processor. The processing circuitry is configured to process a sequence of images to recognize an object in the images, the recognition of the object based upon a hierarchical model. In another example, a method includes determining input data from a plurality of overlapping pixel patches of a video image; determining a plurality of corresponding states based at least in part upon the input data and an over-complete dictionary of filters; and determining a cause based at least in part upon the plurality of corresponding states. The cause may be used to identify an object in the video image.

First claim

Therefore, at least the following is claimed:

1. A system, comprising:
processing circuitry including a processor, the processing circuitry configured to process a sequence of images to recognize an object in the sequence of images, the recognition of the object based upon a hierarchical model, where:
a first level of the hierarchical model includes:
a state estimator configured to determine state data associated with one image of the sequence of images based at least in part upon input data from the one image, the input data comprising a plurality of vectors associated with corresponding overlapping pixel patches extracted from the one image; and
a parameter estimator configured to determine parameters of the state estimator of the first level based at least in part upon the input data from the one image; and
a second level of the hierarchical model includes:
a state estimator configured to determine state data associated with the one image based at least in part upon input data from the first level of the hierarchical model and provide a top-down prediction to the state estimator of the first level; and
a parameter estimator configured to determine parameters of the state estimator of the second level based at least in part upon the input data from the first level.

2. The system of claim 1 , wherein the first level of the hierarchical model receives input data from another image of the sequence of images and provides output data as input data to the second level of the hierarchical model.

3. The system of claim 1 , wherein the hierarchical model is a distributed hierarchical model.

4. A system, comprising:
a first image processing layer of a hierarchical model, including:
sparse inference input circuitry configured to determine a plurality of states based at least in part upon input data associated with a video image; and
sparse inference output circuitry configured to determine a cause associated with the video image, the cause based at least in part upon the plurality of states determined by the sparse inference input circuitry of the first image processing layer; and
a second image processing layer of the hierarchical model, including:
sparse inference input circuitry configured to determine at least one state based at least in part upon input data comprising the cause determined by the sparse inference output circuitry of the first image processing layer; and
sparse inference output circuitry configured to determine a second cause associated with the video image, the second cause based at least in part upon the at least one state determined by the sparse inference input circuitry of the second image processing layer; where the sparse inference output circuitry of the second image processing layer is configured to receive a top-down prediction from sparse inference output circuitry of a third image processing layer.

5. The system of claim 4 , wherein the sparse inference input circuitry of the second image processing layer is configured to provide a top-down prediction to the sparse inference output circuitry of the first image processing layer.

6. The system of claim 4 , wherein the input data comprises a plurality of vectors associated with corresponding overlapping pixel patches extracted from the video image.

7. The system of claim 4 , wherein a first state of the plurality of states are based at least in part upon a state-space prediction corresponding to a preceding first state.

8. The system of claim 4 , wherein the input data associated with the video image comprises causes determined by a second image processing layer.

9. The system of claim 4 , wherein the sparse inference input circuitry of the first image processing layer infers the state based at least in part upon an over-complete dictionary of filters.

10. The system of claim 4 , wherein the sparse inference output circuitry of the first image processing layer infers the cause based at least in part upon an invariant matrix.

11. The system of claim 4 , wherein the sparse inference output circuitry of the first image processing layer is configured to provide a sparsity parameter to the sparse inference input circuitry of the first image processing layer.

12. A method, comprising:
determining input data from a plurality of overlapping pixel patches of a video image;
determining a plurality of corresponding states based at least in part upon the input data associated with the video image and an over-complete dictionary of filters;
determining a first cause associated with a first layer of a hierarchical model based at least in part upon the plurality of corresponding states;
determining at least one state based at least in part upon input data comprising the first cause, where the first cause is modulated based upon a top-down prediction associated with a second cause associated with a second layer of the hierarchical model; and
determining the second cause associated with the second layer of the hierarchical model, the second cause based at least in part upon the at least one state.

13. The method of claim 12 , further comprising:
determining a sparsity parameter based at least in part upon the plurality of corresponding states; and
determining a plurality of subsequent states based at least in part upon the sparsity parameter.

14. The method of claim 12 , wherein the first cause is modulated based upon a top-down prediction associated with a preceding cause.

Assignees

• Univ Florida

Inventors

• Chalasani Rakesh
• Principe Jose C

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2136

  based on sparsity criteria, e.g. with an overcomplete basis · CPC title

• G06V10/454 Primary

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title