Invariant-based dimensional reduction of object recognition features, systems and methods

Patent metadata

Field	Value
Publication number	US-11188786-B2
Application number	US-201916538800-A
Country	US
Kind code	B2
Filing date	Aug 12, 2019
Priority date	Feb 19, 2014
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

A sensor data processing system and method is described. Contemplated systems and methods derive a first recognition trait of an object from a first data set that represents the object in a first environmental state. A second recognition trait of the object is then derived from a second data set that represents the object in a second environmental state. The sensor data processing systems and methods then identifies a mapping of elements of the first and second recognition traits in a new representation space. The mapping of elements satisfies a variance criterion for corresponding elements, which allows the mapping to be used for object recognition. The sensor data processing systems and methods described herein provide new object recognition techniques that are computationally efficient and can be performed in real-time by the mobile phone technology that is currently available.

First claim

What is claimed is:

1. A graphic image processing method using at least one memory and one or more processors in communication with the at least one memory, the method comprising:
obtaining a sensor data set representative of at least one graphic image, the sensor data set comprising image data obtained from a sensor within an environment having one or more environmental parameters that correspond to one or more environmental attributes;
deriving a recognition trait comprising a plurality of elements from the sensor data set according to a trait extraction algorithm;
recognizing a game object as a real-world object by mapping the recognition trait to a game object trait vocabulary in a representation space, wherein the mapping of the plurality of elements in the representation space satisfies trait element variance criteria among corresponding elements in the recognition trait and an associated game object trait mapped to the recognition trait, and comprises an invariant property or a variant property of at least one of the elements of the recognition trait with respect to adjusted environmental parameters or adjusted environmental attributes, and the game object trait vocabulary comprises a plurality of cells each having an image descriptor representing a recognition trait and indexed according to an identifier; and
classifying the game object or the real-world object based on the recognition trait.

2. The method of claim 1 , wherein the game object trait vocabulary is generated based on descriptor clusters derived from at least the sensor data set.

3. The method of claim 1 , wherein the one or more environmental parameters include at least one of a duration or length of time, a sampling or analysis frequency, and a distortion of time via slowing down or speeding up sensor data playback.

4. The method of claim 1 , wherein the one or more environmental parameters include one or more lighting properties.

5. The method of claim 4 , wherein the one or more lighting properties include one or more of light color and background color.

6. The method of claim 1 , wherein the one or more environmental parameters include one or more object properties.

7. The method of claim 6 , wherein the one or more object properties comprise object color.

8. The method of claim 1 , wherein the one or more environmental parameters include at least one of object movement, object direction or object phase.

9. The method of claim 1 , wherein the sensor data set comprises accelerometer data.

10. The method of claim 1 , wherein the sensor data set comprises video data.

11. The method of claim 10 , wherein the one or more environmental parameters include a video frame rate.

12. The method of claim 1 , wherein the real-world object comprises a person.

13. The method of claim 1 , further comprising classifying the real-world object as a type of object based on the recognition trait.

14. The method of claim 1 , further comprising tracking a location of the recognition trait across multiple environmental states.

15. The method of claim 1 , wherein the environment comprises a virtual or simulated environment.

16. The method of claim 1 , wherein the trait extraction algorithm comprises at least one of an image processing algorithm, a classification algorithm, an object recognition algorithm, and an edge-based recognition technique.

17. The method of claim 16 , wherein the image processing algorithm comprises at least one of a SIFT, FAST, FREAK, BRIEF, ORB, BRISK, GLOH, SURF, vSLAM, SLAM, BURST, and DAISY image processing algorithm.

18. The method of claim 1 wherein the recognition trait comprises an image descriptor.

19. The method of claim 18 , wherein the plurality of elements comprises dimensions of the image descriptor.

20. The method of claim 1 , wherein the recognition trait comprises a cluster of descriptors.

21. The method of claim 1 , further comprising classifying the game object as a type of real-world object.

22. The method of claim 1 , wherein the mapping defines a new game object trait vocabulary.

23. The method of claim 1 , wherein the mapping includes a dimensionality reduction of the plurality of elements associated with the invariant property of one or more elements.

24. A graphic image processing system comprising:
at least one memory storing graphic image processing instructions; and
one or more processors in communication with the at least one memory such that, when executed, the graphic image processing instructions cause the processor to:
obtain a sensor data set representative of at least one graphic image, the sensor data set comprising image data obtained from a sensor within an environment having one or more environmental parameters that correspond to one or more environmental attributes;
derive a recognition trait comprising a plurality of elements from the sensor data set according to a trait extraction algorithm;
recognize the game object as a real-world object by mapping the recognition trait to a game object trait vocabulary in a representation space, wherein the mapping of the plurality of elements in the representation space satisfies trait element variance criteria among corresponding elements in the recognition trait and an associated game object trait mapped to the recognition trait, and comprises an invariant property or a variant property of at least one of the elements of the recognition trait with respect to adjusted environmental parameters or adjusted environmental attributes, and the game object trait vocabulary comprises a plurality of cells each having an image descriptor representing a recognition trait and indexed according to an identifier; and
classify the game object or the real-world object based on the recognition trait.

25. A computer program product embedded in a non-transitory computer-readable medium comprising instructions for graphic image processing, which, when executed, configure one or more processors to perform a method comprising:
obtaining a sensor data set representative of at least one graphic image, the sensor data set comprising image data obtained from a sensor within an environment having one or more environmental parameters that correspond to one or more environmental attributes;
deriving a recognition trait comprising a plurality of elements from the sensor data set according to a trait extraction algorithm;
recognize the game object as a real-world object by mapping the recognition trait to a game object trait vocabulary in a representation space, wherein the mapping of the plurality of elements in the representation space satisfies trait element variance criteria among corresponding elements in the recognition trait and an associated game object trait mapped to the recognition trait, and comprises an invariant property or a variant property of at least one of the elements of the recognition trait with respect to adjusted environmental parameters or adjusted environmental attributes, and the game object trait vocabulary comprises a plurality of cells each having an image descriptor representing a recognition trait and indexed according to an identifier; and
classify the game object or the real-world object based on the recognition trait.

Assignees

• Nant Holdings Ip Llc

Inventors

• Wnuk Kamil
• Sudol Jeremi
• Song Bing
• Siddiqui Matheen
• Mckinnon David

Classifications

• G06V10/771

  Feature selection, e.g. selecting representative features from a multi-dimensional feature space · CPC title

• G06F18/2113

  by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation · CPC title

• G06K9/623

  Physics · mapped topic

• G06K9/4633

  Physics · mapped topic

• G06K9/4671

  Physics · mapped topic