Fast recognition algorithm processing, systems and methods

Patent metadata

Field	Value
Publication number	US-9690991-B2
Application number	US-201615336754-A
Country	US
Kind code	B2
Filing date	Oct 27, 2016
Priority date	Jul 19, 2013
Publication date	Jun 27, 2017
Grant date	Jun 27, 2017

Abstract

Systems and methods of quickly recognizing or differentiating many objects are presented. Contemplated systems include an object model database storing recognition models associated with known modeled objects. The object identifiers can be indexed in the object model database based on recognition features derived from key frames of the modeled object. Such objects are recognized by a recognition engine at a later time. The recognition engine can construct a recognition strategy based on a current context where the recognition strategy includes rules for executing one or more recognition algorithms on a digital representation of a scene. The recognition engine can recognize an object from the object model database, and then attempt to identify key frame bundles that are contextually relevant, which can then be used to track the object or to query a content database for content information.

First claim

What is claimed is:

1. A mobile device comprising:
at least one sensor;
at least one display;
a tangible, non-transitory computer readable memory storing software instructions; and
at least one hardware processor coupled with the memory and the sensor, and configurable, upon execution of the software instructions, to:
capture, via the at least one sensor, a digital representation a scene;
obtain access to contextually relevant key frame bundles based on a context derived from the digital representation, wherein the contextually relevant key

frame bundles include recognition features associated with modeled features of at least
one known object;
recognize at least one scene object in the scene as the at least one known object according to at least one recognition algorithm and as a function of the recognition features and the digital representation;
obtain access to augmented reality (AR) content associated with the at least one known object, wherein the AR content comprises an object mask generated from an object model of the at least one known object;
anchor the AR content to an anchor point on the at least one scene object as a function of the modeled features corresponding to the recognition features used to

recognize the at least one scene object; and
render the AR content on the display in relation to the anchor point on the at least one scene object.

2. The mobile device of claim 1 , wherein the at least one hardware processor is further configurable to track the at least one scene object in the digital representation in real-time as a function of the recognition features.

3. The mobile device of claim 2 , wherein the at least one hardware processor is further configurable to track at least ten recognized scene objects in the digital representation in real-time as a function of corresponding recognition features.

4. The mobile device of claim 2 , wherein the at least one hardware processor is further configurable to track the at least one scene object at a frame rate of a video rendering of the AR content on the display.

5. The mobile device of claim 4 , wherein the frame rate is at least 30 frames per second.

6. The mobile device of claim 1 , wherein the digital representation comprises at least one of image data, video data, audio data, location data, biometric data, tactile data, time data, temperature data, and accelerometer data.

7. The mobile device of claim 1 , wherein the AR content is rendered as an overlay over video data captured as part of the digital representation.

8. The mobile device of claim 1 , wherein the AR content further comprises at least one of a link, a URL, image data, video data, audio data, augmented reality data, social media data, product data, text data, object data, object model data, game data, news data, and multimedia data.

9. The mobile device of claim 1 , wherein the
object mask comprises an inverse object mask.

10. The mobile device of claim 1 , wherein the context is based on at least one of a time, a location, an orientation, a personal preference, a weather condition, a proximity, a temperature, and an acceleration.

11. The mobile device of claim 1 , wherein the context includes at least one of a contextual device state, a shopping context, a commute context, a birthday context, a gaming context, and a non-visual context.

12. The mobile device of claim 1 , wherein the contextually relevant key frame bundles include non-imaging modeled object properties of the at least one known object.

13. The mobile device of claim 12 , wherein the AR content is rendered as a function of the non-imaging modeled object properties.

14. The mobile device of claim 12 , wherein the non-imaging modeled object properties include at least one of a material property, a mechanical property, a magnetic property, an optical property, a thermal property, an acoustic property, a chemical property, and an electrical property.

15. The mobile device of claim 1 , wherein the anchor point corresponds to at least one of the modeled features when the at least one scene object is deformed relative to the at least one known object.

16. The mobile device of claim 1 , wherein the scene comprises a 3D scene.

17. The mobile device of claim 1 , wherein the scene comprises a video scene.

18. The mobile device of claim 1 , wherein the scene comprises a real-world environment.

19. The mobile device of claim 1 , wherein the scene comprises a virtual setting.

20. The mobile device of claim 1 , wherein the scene comprises a non-visual environment.

21. The mobile device of claim 1 , wherein the at least one known object comprises at least one of a building, an automobile, a person, a face, a game player, a game character, a lamp post, and a vehicle.

22. The mobile device of claim 1 , wherein the mobile device is one of a cell phone, a tablet, a game console, a game, an appliance, a robot, a vehicle, a toy, a hand-held game system, a personal data assistant, a medical device, a UAV, and a phablet.

Assignees

• Nant Holdings Ip Llc

Inventors

• Wnuk Kamil
• Song Bing
• Siddiqui Matheen
• Mckinnon David
• Sudol Jeremi
• Soon-Shiong Patrick
• Dialameh Orang

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/20004

  Adaptive image processing · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G16H30/40

  for processing medical images, e.g. editing · CPC title

• G06T7/0014

  using an image reference approach · CPC title