Methods for object recognition and related arrangements

Patent metadata

Field	Value
Publication number	US-9269022-B2
Application number	US-201414251229-A
Country	US
Kind code	B2
Filing date	Apr 11, 2014
Priority date	Apr 11, 2013
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Methods and arrangements involving portable user devices such smartphones and wearable electronic devices are disclosed, as well as other devices and sensors distributed within an ambient environment. Some arrangements enable a user to perform an object recognition process in a computationally- and time-efficient manner. Other arrangements enable users and other entities to, either individually or cooperatively, register or enroll physical objects into one or more object registries on which an object recognition process can be performed. Still other arrangements enable users and other entities to, either individually or cooperatively, associate registered or enrolled objects with one or more items of metadata. A great variety of other features and arrangements are also detailed.

First claim

We claim:

1. A method employing one or more computer processors to perform acts including:
storing, within an object registry, model data and object metadata corresponding to a plurality of physical reference objects, the model data for each physical reference object including data characterizing plural non-coplanar surface regions of different extents and locations, and hybrid-P feature data, the model data also including, for each of several different physical reference objects, multiple sets of feature information, each set of feature information being associated with a particular viewpoint towards a physical reference object;
obtaining query data representing a physical object-of-interest, wherein generation of the query data is initiated by a user, and said query data includes object profile data representing an edge of a silhouette of the physical object-of-interest;
performing an object recognition process on the query data, the object recognition process including processing the query data, in conjunction with the stored model data, to determine whether the object-of-interest corresponds to any of the plurality of physical reference objects, said object recognition process including identifying plural sets of said feature information that may correspond to the query data, thereby identifying a first candidate set of plural physical reference objects that possibly match said physical object-of-interest, and performing a clustering operation on viewpoints associated with said identified plural sets of feature information, to determine a preliminary candidate viewpoint towards a matching physical reference object;
the object recognition process further including, for each of said first candidate set of physical reference objects, obtaining reference object profile data corresponding to said preliminary candidate viewpoint, and for one or more additional viewpoints; and
performing a profile matching operation to identify certain of said obtained reference object profile data that correspond to the object profile data representing the physical object-of-interest, thereby identifying a second candidate set of physical reference objects that possibly match said physical object-of-interest, said second candidate set being smaller than said first candidate set; and
upon determining that the object-of-interest corresponds to at least one of the physical reference objects, transmitting a result to a user device associated with the user, the result including object metadata associated with the at least one of the physical reference objects determined to correspond to the object-of-interest;
wherein the hybrid-P feature data is based on an accumulation of multiple profiles of the reference object from multiple viewpoints.

2. The method of claim 1 , wherein the query data is obtained from the user device.

3. The method of claim 1 , wherein the query data is obtained from a camera-equipped ambient device distinct from said user device.

4. The method of claim 1 , wherein said object metadata associated with at least one of the physical reference objects determined to correspond to the object-of-interest includes object identifying information identifying the reference object.

5. The method of claim 1 , further comprising identifying model data corresponding to a sub-set of the plurality of physical reference objects based on auxiliary information,
wherein processing the model data comprises processing the identified model data to determine whether the object-of-interest corresponds to any physical reference object within said sub-set of the plurality of physical reference objects.

6. The method of claim 1 wherein said act of performing an object recognition process includes three stages or operation, (b), (c) and (d), wherein stage (c) is performed after stage (b) and before stage (d), and wherein:
stage (b) is based on said hybrid P-feature data;
stage (c) is based on M-feature data; and
stage (d) is based on I-feature data.

7. The method of claim 6 wherein stage (c) is based on the hybrid P-feature data, as well as on said M-feature data.

8. The method of claim 6 wherein stage (d) is based on the M-feature data and said hybrid P-feature data, as well as on said I-feature data.

9. The method of claim 6 wherein the object recognition process includes a stage (a), wherein stage (a) is performed before stage (b), and wherein stage (a) includes match searching based on color histogram data.

10. The method of claim 1 wherein performing the object recognition process includes receiving, at the user device, stored model data transmitted from a retail store, enabling the user device to recognize retail objects at said retail store.

11. The method of claim 1 in which said act of obtaining query data includes obtaining data from first and second cameras, the first camera comprising part of the user device, the second camera comprising an ambient camera.

12. The method of claim 11 in which said generation of the query data is initiated by a user gesture indicating the physical object-of-interest, wherein the method includes, in response to said gesture, and in response to sampled location or user device orientation information, identifying an ambient camera to serve as said second camera—from among plural ambient cameras.

13. The method of claim 1 that includes sensing object data from one of said plurality of physical reference objects while illuminating such physical reference object with an optical system that produces collimated illumination, said physical reference object having a maximum dimension of N centimeters, and said optical system having an aperture greater than N centimeters.

14. The method of claim 1 wherein said optical system includes a light source that is tunable across the visible light spectrum.

15. The method of claim 1 wherein said model data for one of said physical reference objects defines more than 1 million planar surface components in a mesh form.

16. The method of claim 1 wherein said object recognition process includes sleuthing a projective viewpoint of the query data relative to the model data for one of said physical reference objects.

17. The method of claim 1 wherein said feature information comprises color histogram information.

18. The method of claim 1 that further includes:
for each physical reference object in said second set, determining a viewpoint that most closely corresponds to said query data, and generating a match score employing match metrics for profile, Morse, and image features associated with said reference object and said determined viewpoint; and
identifying, as a final match to said physical object of interest, a physical reference object for which said match score is best.

19. The method of claim 18 in which said match score takes the form of a polynomial equation:
aKp d +bKi e +cKm f

where a, b, and c, are weighting factors, Kp, Ki and Km are match-metrics for the profile, image and Morse features, and d, e and f are corresponding exponential factors.

20. The method of claim 1 in which the act of obtaining query data includes obtaining query data across five different spectral bands.

21. The method of claim 1 in which said model data includes data characterizing reflectance for each of said surface regions.

22. The method of claim 1 in which:
the act of obtaining query data includes applying a visual saliency model to imagery of the physical object-of-interest to identify visually-salie

Assignees

• Digimarc Corp

Inventors

• Rhoads Geoffrey B
• Bai Yang
• Rodriguez Tony F
• Rogers Eliot
• Sharma Ravi K
• Lord John D
• Long Scott
• Macintosh Brian T
• Eaton Kurt M

Classifications

• G06V10/772 Primary

  Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title