Context aware localization, mapping, and tracking

Patent metadata

Field	Value
Publication number	US-9367811-B2
Application number	US-201313842928-A
Country	US
Kind code	B2
Filing date	Mar 15, 2013
Priority date	Mar 15, 2013
Publication date	Jun 14, 2016
Grant date	Jun 14, 2016

Abstract

Exemplary methods, apparatuses, and systems infer a context of a user or device. A computer vision parameter is configured according to the inferred context. Performing a computer vision task, in accordance with the configured computer vision parameter. The computer vision task may by at least one of: a visual mapping of an environment of the device, a visual localization of the device or an object within the environment of the device, or a visual tracking of the device within the environment of the device.

First claim

What is claimed is:

1. A machine-implemented method for performing a computer vision task, the method comprising:
inferring a context of a user or device, wherein the context comprises one or more of motion, activity, environment, or location, or any combination thereof;
configuring a computer vision task according to the inferred context; and
performing the computer vision task, wherein the computer vision task comprises at least one of:
mapping of an environment of the device,
localizing of the device or an object within the environment of the device,
tracking of the device within the environment of the device, or
any combination thereof.

2. The machine-implemented method of claim 1 , wherein the inferring a context comprises reading sensor data to infer the context of the user or the device, wherein the sensor data is from one or more of: Bluetooth sensor, microphone, camera(s), global positioning sensor (GPS), WiFi, proximity sensor, temperature sensor, barometric (i.e., pressure) sensor, ambient light sensor (ALS), red-green-blue (RGB) color sensor, ultra-violet (UV) sensor, UV-A sensor, UV-B sensor, fingerprint sensor, touch sensor, accelerometer, gyro, compass, magnetometer, or any combination thereof.

3. The machine-implemented method of claim 1 , wherein the inferring a context comprises reading application data to infer the context of the user or the device, wherein the application data is from one or more of: calendar, geo tagging, social media data, battery, time of day, or any combination thereof.

4. The machine-implemented method of claim 1 , wherein the configuring the computer vision task includes one or more of:
selecting parameters of an algorithm associated with the task;
selecting a prior model as inputs to the algorithm associated with the task;
selecting an approach to solve the problem based on environment constants; or any combination thereof.

5. The machine-implemented method of claim 4 , wherein the configuring the computer vision task includes one or more of:
using edge based methods;
using point based methods;
using rectangle based methods; or
any combination thereof.

6. The machine-implemented method of claim 1 , wherein the context is an indoor context and the computer vision task uses information associated with the indoor context to modify the computer vision task configuration for use with one or more of: an aligned rectangle coordinate system, bounded interior scene dimensions, predetermined objects, indoor structural features, or any combination thereof.

7. The machine-implemented method of claim 1 , wherein the context is an outdoor context and the computer vision task uses information associated with the outdoor context to modify the computer vision task configuration for use with one or more of: dynamic world modeling, predetermined outdoor structural features, distant features, a panoramic model or tracker, images from a ground facing camera, or any combination thereof.

8. The machine-implemented method of claim 1 , wherein the context is a cluttered context and the computer vision task uses information associated with the cluttered context to modify the computer vision task configuration for use with one or more of: feature point calculations, increasing a keyframe rate, or any combination thereof.

9. The machine-implemented method of claim 1 , wherein the context is an uncluttered context and the computer vision task uses information associated with the uncluttered context to modify the computer vision task for use with one or more of:
decreasing a keyframe rate,
using one or more features of: lines, vanishing points, rectangles, or any combination thereof, or
any combination thereof.

10. The machine-implemented method of claim 1 , wherein when the context is an excited context, the computer vision task increases a number of tracked features, and wherein when the context is an unexcited context the computer vision task decreases the number of tracked features.

11. The machine-implemented method of claim 1 , wherein the context is a dynamic context and the computer vision task uses information associated with the dynamic context to modify the computer vision task configuration for use with one or more of:
delaying performing the computer vision task until the context is determined to be static,
selecting one of a plurality of camera sensors to capture a static portion of the dynamic environment and performing the computer vision task using the selected camera sensor, or
any combination thereof.

12. The machine-implemented method of claim 1 , wherein the context is a combination of two or more of: a location context, an environment context, an activity context, a motion context, or any combination thereof.

13. The machine-implemented method of claim 1 , further comprising:
providing, for the computer vision task, a suggested camera direction or viewpoint based on the inferred context.

14. A machine readable non-transitory storage medium containing executable program instructions which cause a data processing device to perform a method for performing a computer vision task, the method comprising:
inferring a context of a user or device, wherein the context comprises one or more of: motion, activity, environment, location, or any combination thereof;
configuring a computer vision task according to the inferred context; and
performing the computer vision task, wherein the computer vision task comprises at least one of:
mapping of an environment of the device,
localizing of the device or an object within the environment of the device,
tracking of the device within the environment of the device, or any combination thereof.

15. The medium of claim 14 , wherein the inferring a context comprises reading sensor data to infer the context of the user or the device, wherein the sensor data is from one or more of: Bluetooth sensor, microphone, camera(s), global positioning sensor (GPS), WiFi, proximity sensor, temperature sensor, barometric (i.e., pressure) sensor, ambient light sensor (ALS), red-green-blue (RGB) color sensor, ultra-violet (UV) sensor, UV-A sensor, UV-B sensor, fingerprint sensor, touch sensor, accelerometer, gyro, compass, magnetometer, or any combination thereof.

16. The medium of claim 14 , wherein the inferring a context comprises reading application data to infer the context of the user or the device, wherein the application data is from one or more of: calendar, geo tagging, social media data, battery, time of day, or any combination thereof.

17. The medium of claim 14 , wherein the context comprises one or more of: motion, activity, environment, location, or any combination thereof.

18. The medium of claim 14 , wherein the configuring the computer vision task includes one or more of:
selecting parameters of an algorithm associated with the task;
selecting a prior model as inputs to the algorithm associated with the task;
selecting an approach to solve the problem based on environment constants; or
any combination thereof.

19. The medium of claim 18 , wherein the configuring the computer vision task includes one or more of:
using edge based methods;
using point based methods;
using rectangle based methods; or
any combination thereof.

20. The medium of claim 14 , wherein the context is an indoor context and the computer vision task uses information associated with the indoor context to modify the computer vision task configuration for use with one or more of: an aligned rectangle coordinate system, bounded interior scen

Assignees

• Qualcomm Inc

Inventors

• Ramachandran Mahesh
• Swaminathan Ashwin
• Sadasivam Shankar

Classifications

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06N7/01 Primary

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G01C21/36

  Input/output arrangements for on-board computers · CPC title

• G06N7/005 Primary

  Physics · mapped topic

• G06V10/96

  Management of image or video recognition tasks · CPC title