Method of classifying objects in scenes

Patent metadata

Field	Value
Publication number	US-9501723-B2
Application number	US-201414579950-A
Country	US
Kind code	B2
Filing date	Dec 22, 2014
Priority date	Dec 24, 2013
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

A method for classifying objects in a scene captured by a camera determines a likelihood of first set of states for the objects. Each first set is a classification of one of the objects, and partitions a solution space based on the determined likelihood of the first set of states, each partition representing combinations of the classifications of the objects. The partitioning is applied to a solution space of a second set of states, each partition representing combinations of the classifications of a subset of the objects. The method determines a likelihood of the second set of states for the subset of the objects, each state of the second set of states being a classification of one of the subset of objects, and classifies a subset of objects according to the determined likelihood of the second set of states and the partitioning of the second set of states.

First claim

The invention claimed is:

1. A method of classifying objects in a scene, captured by a camera, the method comprising:
determining a likelihood of first set of states for the objects in the scene, each of the first set of states being a classification of one of the objects in the scene;
partitioning a solution space based on the determined likelihood of the first set of states, each partition of the solution space representing combinations of the classifications of the objects in the scene;
applying the partitioning of the solution space to a solution space of a second set of states, each partition of the solution space of the second set of states representing combinations of the classifications of a subset of objects in the scene;
determining a likelihood of the second set of states for the subset of the objects in the scene, each state of the second set of states being a classification of one of the subset of objects in the scene; and
classifying at least one of the subset of objects in the scene according to the determined likelihood of the second set of states and the partitioning of the solution space of the second set of states.

2. A method of classifying objects in a scene, the method comprising:
determining a likelihood of first set of states for the objects in a first scene representation, each of the first set of states being a classification of one of the objects in the first scene representation;
partitioning a solution space based on the determined likelihood of the first set of states, each partition of the solution space representing combinations of the classifications of the objects in the first scene representation;
applying the partitioning of the solution space to a solution space of a second set of states, each partition of the solution space of the second set of states representing combinations of the classifications of a subset of objects in a second scene representation associated with the scene;
determining a likelihood of the second set of states for the subset of the objects in the scene, each state of the second set of states being a classification of one of the subset of objects in the second scene representation; and
classifying at least one of the subset of objects in the second scene representation according to the determined likelihood of the second set of states and the partitioning of the solution space of the second set of states.

3. A method according to claim 2 wherein the first scene representation is formed from a first image of a first scene, and the second scene representation is formed from a second image of a second scene.

4. A method according to claim 3 wherein the first and second scenes at least partly overlap.

5. A method according to claim 2 wherein the partitions are formed as lists of states, where each list of states is associated with a discrete random variable of the first or second scene representation.

6. A method according to claim 5 wherein the first and second scene representations comprise discrete Markov random field (MRF) representations, and for a first kind of difference between the scene representations, the partitioning makes no changes to the partitions, and the lists of states stored by the partitions are associated with discrete random variables and states of the second scene representation.

7. A method according to claim 6 wherein the first kind of difference comprises difference in values of potential functions of the scene representations.

8. A method according to claim 5 , wherein the first and second scene representations comprise discrete Markov random field (MRF) representations, and for a second kind of difference, a partition is transformed, for each variable for which new states have been added, by adding the new states to the list of states for that variable in the partition.

9. A method according to claim 8 , wherein the new states are added in replacement of an existing state and the list of states includes the state to be replaced, then the state to be replaced is removed from the list of states.

10. A method according to claim 8 wherein the second kind of difference between the scene representations is where the second scene representation comprises new states for variables, the new states being in addition to existing states of those variables or in replacement of existing states of those variables.

11. A method according to claim 5 , wherein the first and second scene representations comprise discrete Markov random field (MRF) representations, and for a third kind of difference, a partition is transformed, for each new variable, by adding all of the possible states of the new variable to the partition, wherein the lists of states for other variables in the partition are unchanged.

12. A method according to claim 11 wherein the third kind of difference comprises the second scene representation containing new variables in addition to the variables of the first scene representation.

13. A method according to claim 5 , wherein the first and second scene representations comprise discrete Markov random field (MRF) representations, and the partitioning is responsive to three kinds of differences between the scene representations, being:
(i) a first kind of difference between the scene representations, the partitioning makes no changes to the partitions, and the lists of states stored by the partitions are associated with discrete random variables and states of the second scene representation, the first kind of difference comprising differences in values of potential functions of the scene representations;
(ii) a second kind of difference, where a partition is transformed, for each variable for which new states have been added, by adding the new states to the list of states for that variable in the partition, wherein the new states are added in replacement of an existing state and the list of states includes the state to be replaced, then the state to be replaced is removed from the list of states, wherein the second kind of difference between the scene representations is where the second scene representation comprises new states for variables, the new states being in addition to existing states of those variables or in replacement of existing states of those variables; and
(iii) a third kind of difference, where a partition is transformed, for each new variable, by adding all of the possible states of the new variable to the partition, wherein the lists of states for other variables in the partition are unchanged, wherein the third kind of difference comprises the second scene representation containing new variables in addition to the variables of the first scene representation.

14. A method according to claim 5 , wherein the first and second scene representations comprise discrete Markov random field (MRF) representations, and the step of applying the partitioning comprises applying the partitioning of a first model derived from the first scene representation to a second model associated with the second scene representation.

15. A method according to claim 5 , wherein the partitioning further comprises storing the partitions in a priority queue ordered according to the first scene representation and the applying comprises transforming the stored partitions into partitions of the second scene representation.

16. A method according to claim 15 , further comprising adding a new variable to each partition in the priority queue such that partitions of the second scene representation derived from the first scene representation are identifiable therefrom.

17. A non-transitory computer readable storage

Assignees

• Canon Kk

Inventors

• Kim Min Sub

Classifications

• G06V10/84

  using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks · CPC title

• G06F18/29

  Graphical models, e.g. Bayesian networks · CPC title

• G06V20/00 Primary

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title

• G06K9/6296 Primary

  Physics · mapped topic

• G06K9/00624

  Physics · mapped topic