Odometry feature matching

Patent metadata

Field	Value
Publication number	US-9437000-B2
Application number	US-201414185616-A
Country	US
Kind code	B2
Filing date	Feb 20, 2014
Priority date	Feb 20, 2014
Publication date	Sep 6, 2016
Grant date	Sep 6, 2016

Abstract

Methods and systems for determining features of interest for following within various frames of data received from multiple sensors of a device are disclosed. An example method may include receiving data from a plurality of sensors of a device. The method may also include determining, based on the data, motion data that is indicative of a movement of the device in an environment. The method may also include as the device moves in the environment, receiving image data from a camera of the device. The method may additionally include selecting, based at least in part on the motion data, features in the image data for feature-following. The method may further include estimating one or more of a position of the device or a velocity of the device in the environment as supported by the data from the plurality of sensors and feature-following of the selected features in the images.

First claim

What is claimed is:

1. A method comprising:
receiving, using a processor, data from a plurality of sensors of a device;
determining, based on the data, motion data that is indicative of a movement of the device in an environment;
as the device moves in the environment, receiving image data from a camera of the device;
selecting, based at least in part on the motion data, features in the image data for feature-following; and
estimating one or more of a position of the device or a velocity of the device in the environment as supported by the data from the plurality of sensors and by feature-following of the selected features in the image data:
wherein the plurality of sensors includes an inertial measurement unit (IMU),
wherein the motion data includes data received from the IMU that is indicative of a direction of gravity being applied to the device, and
wherein selecting features in the image data for feature-following comprise:
identifying one or more vertical lines in the image data as a line that is parallel to the direction of gravity or one or more horizontal lines in the image data as a line that is perpendicular to the direction of gravity; and
providing the identified one or more vertical lines or the identified one or more horizontal lines as a given feature for feature-following.

2. The method of claim 1 ,
wherein the plurality of sensors further includes a depth sensor,
wherein the motion data includes data received from the depth sensor that is indicative of a plurality of depths associated with the movement of the device, and
wherein selecting features in the image data for feature-following further comprises:
determining, based on respective depths of the plurality of depths associated with the movement of the device, a plurality of cells in the image data that are divided by respective depths of the plurality of depths; and
based on a given cell of the plurality of cells including a number of features less than a threshold number of features, selecting features for feature-following that correspond to the given cell.

3. The method of claim 1 ,
wherein selecting features in the image data for feature tracking further comprises:
determining, based on the image data, a plurality of sections of the image data; and
based on a given section of the plurality of sections including a number of features less than a threshold number of features, selecting features for feature-following that correspond to the given section.

4. The method of claim 3 , wherein respective sections of the plurality of sections are defined based on quadrants associated with the image data.

5. The method of claim 3 ,
wherein the camera comprises a wide-angle view camera, and
wherein respective sections of the plurality of sections are defined based on angles associated with the image data.

6. The method of claim 1 , further comprising:
receiving, using the processor, environmental data from the plurality of sensors of the device, wherein the environmental data includes shape data that defines one or more shapes of objects in the environment; and
selecting, based at least in part on the shape data, the features in the image data for feature-following.

7. The method of claim 1 , further comprising:
determining, based on the motion data, an uncertainty of movement of the device that defines a level of uncertainty of a portion of the movement of the device; and
selecting, based on the uncertainly of movement of the device, the features in the image data for feature-following.

8. The method of claim 7 ,
wherein the uncertainty of movement of the device comprises an uncertainty of a roll or a pitch of the device, and
wherein selecting the features in the image data for feature-following comprises selecting the identified one or more horizontal lines for feature-following.

9. The method of claim 7 ,
wherein the uncertainty of movement of the device comprises an uncertainty of a yaw of the device, and
wherein selecting the features in the image data for feature-following comprises selecting the identified one or more vertical lines for feature-following.

10. A non-transitory computer readable memory configured to store instructions that, when executed by a device, cause the device to perform functions comprising:
receiving, at the device, data from a plurality of sensors of the device;
determining, based on the data, motion data that is indicative of a movement of the device in an environment;
as the device moves in the environment, receiving image data from a camera of the device;
selecting, based at least in part on the motion data, features in the image data for feature-following;
estimating one or more of a position of the device or a velocity of the device in the environment as supported by the data from the plurality of sensors and by feature-following of the select features in the image data; and
wherein the plurality of sensors includes an inertial measurement unit (IMU),
wherein the motion data includes data received from the IMU that is indicative of a direction of gravity being applied to the device, and
wherein selecting features in the image data for feature-following comprise:
identifying one or more vertical lines in the image data as a line that is parallel to the direction of gravity or one or more horizontal lines in the image data as a line that is perpendicular to the direction of gravity; and
providing the identified one or more vertical lines or the identified one or more horizontal lines as a given feature for feature-following.

11. The non-transitory computer readable memory of claim 10 ,
wherein the plurality of sensors includes a depth sensor,
wherein the motion data further includes data received from the depth sensor that is indicative of a plurality of depths associated with the movement of the device, and
wherein selecting features in the image data for feature-following further comprises:
determining, based on respective depths of the plurality of depths associated with the movement of the device, a plurality of cells in the image data that are divided by respective depths of the plurality of depths; and
based on a given cell of the plurality of cells including a number of features less than a threshold number of features, selecting features for feature-following that correspond to the given cell.

12. The non-transitory computer readable memory of claim 10 ,
wherein selecting features in the image data for feature-following further comprises:
determining, based on the image data, a plurality of sections of the image data; and
based on a given section of the plurality of sections including a number of features less than a threshold number of features, selecting features for feature-following that correspond to the given section.

13. The non-transitory computer readable memory of claim 10 , further comprising:
receiving, using the device, environmental data from the plurality of sensors of the device, wherein the environmental data includes shape data that defines one or more shapes of objects in the environment; and
selecting, based at least in part on the shape data, the features in the image data for feature-following.

14. The non-transitory computer readable memory of claim 10 , further comprising:
determining, based on the motion data, an uncertainty of movement of the device that defines a level of uncertainty of a portion of the movement of the device; and
selecting, based on the uncertainly of movement of the device, the features in the image data for feature-following.

15. A device comprising:
one or more processors; and

Assignees

• Google Inc

Inventors

• Lee Johnny
• Hesch Joel

Classifications

• G06T7/0042 Primary

  Physics · mapped topic

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06K9/00744

  Physics · mapped topic

• H04W4/023 Primary

  using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds · CPC title