Methods and systems for mobile-agent navigation

Patent metadata

Field	Value
Publication number	US-9969337-B2
Application number	US-201514941691-A
Country	US
Kind code	B2
Filing date	Nov 16, 2015
Priority date	Sep 3, 2014
Publication date	May 15, 2018
Grant date	May 15, 2018

Abstract

Feature detection may be performed on an image. After feature descriptors for each detected feature are computed, feature matching between feature descriptors for the current image and for a key image frame is performed. If a sufficient number of good matches are identified, key points associated with the feature correspondences may be projected from image coordinates to world coordinates. A distance, in the world coordinate frame, between each feature correspondence may he computed. When the computed distances indicate sufficient movement of the mobile agent to ensure accurate motion estimation, a motion estimate may he computed from the pairs of world coordinates associated with the feature correspondences. A current camera pose in a global coordinate frame may be generated. A motion trajectory may then be determined and feature descriptors for the key image may be updated to the feature descriptors for the current image frame.

First claim

What is claimed is:

1. A method for estimating motion, said method comprising:
performing feature detection on a current image frame to identify a plurality of regions in said current image frame, wherein each region in said plurality of regions is associated with a key point in an image coordinate frame;
computing a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said current image frame;
performing feature matching between said plurality of feature descriptors for said current image frame and a plurality of feature descriptors computed for a key image frame, thereby producing a plurality of feature correspondences; and
when a number of feature correspondences in said plurality of feature correspondences meets a sufficiency criterion:
for each feature correspondence in said plurality of feature correspondences, computing a distance between said corresponding features; and
when said plurality of distances meet a first criterion:
for each feature correspondence in said plurality of feature correspondences, projecting said associated key points from said image coordinate frames to a world coordinate frame, thereby producing a plurality of pairs of world coordinates;
computing a motion estimate from said plurality of pairs of world coordinates;
generating a current camera pose in a global coordinate frame;
determining a motion trajectory from said current camera pose; and
updating said plurality of feature descriptors computed for said key image frame to said plurality of feature descriptors for said current image frame.

2. A method as described in claim 1 , wherein said motion estimate comprises a rotation matrix and a translation vector.

3. A method as described in claim 2 , wherein said rotation matrix is associated with an in-plane rotation.

4. A method as described in claim 1 , wherein said current image frame is received from a camera rigidly mounted on a mobile agent.

5. A method as described in claim 1 , wherein said performing feature detection comprises using a feature detector selected from the group consisting of an edge detector, a corner detector, a blob detector, a peak detector, a SIFT key-point detector, a SURF key-point detector, a FAST key-point detector, a grid-based FAST key-point detector, an ORB key-point detector, a MSER key-point detector, a BRIEF key-point detector, a BRISK key-point detector, a FREAK key-point detector and a STAR key-point detector.

6. A method as described in claim 1 , wherein said feature descriptor is a feature descriptor selected from the group consisting of a block of pixel values, a block of normalized pixel values, a block of gradient values, a block of adjusted pixel values, a SIFT feature descriptor, a SURF feature descriptor, an ORB feature descriptor, a BRIEF feature descriptor, a BRISK feature descriptor and a FREAK feature descriptor.

7. A method as described in claim 1 , wherein said performing feature matching comprises a bitwise comparison of a first feature descriptor and a second feature descriptor.

8. A method as described in claim 1 , wherein said performing feature matching comprises pruning a plurality of candidate matches.

9. A method as described in claim 1 , wherein said computing a motion estimate comprises using a motion estimator selected from the group consisting of an Orthogonal-Procrustes-Analysis-based motion estimator and an Absolute-Orientation-based motion estimator.

10. A method as described in claim 1 , wherein said computing a motion estimate comprises rejecting an outlier in said plurality of feature correspondences.

11. A method as described in claim 10 , wherein said rejecting comprises performing RANSAC motion estimation.

12. A method as described in claim 1 , wherein said generating a current camera pose comprises concatenating said estimated motion to a previous camera pose.

13. A mobile agent comprising:
a rigidly mounted camera;
a processor component; and
a non-transitory computer-readable medium encoded with a computer program code for causing said processor component to execute a method for estimating motion, said method comprising:
performing feature detection on a current image frame to identify a plurality of regions in said current image frame, wherein each region in said plurality of regions is associated with a key point in an image coordinate frame;
computing a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said current image frame;
performing feature matching between said plurality of feature descriptors for said current image frame and a plurality of feature descriptors computed for a key image frame, thereby producing a plurality of feature correspondences; and
when there are a sufficient number of feature correspondences in said plurality of feature correspondences:
for each feature correspondence in said plurality of feature correspondences, computing a distancing between said corresponding features; and
when said plurality of distances meet a first criterion:
for each feature correspondence in said plurality of feature correspondences, projecting said associated key points from said image coordinate frames to a world coordinate frame, thereby producing a plurality of pairs of world coordinates;
computing a motion estimate from said plurality of pairs of world coordinates;
generating a current camera pose in a global coordinate frame;
determining a motion trajectory from said current camera pose; and
updating said plurality of feature descriptors computed for said key image frame to said plurality of feature descriptors for said current image frame.

14. A mobile agent as described in claim 13 , wherein said camera is rigidly mounted with respect to a ground plane.

15. A mobile agent as described in claim 13 , wherein
said camera is mounted on a front end or a back end of said mobile agent and above a body chassis of said mobile agent; and
said when said camera is mounted on said front end of said mobile agent, said camera is tilted to have a field of view of a ground path directly in front of said mobile agent and when said camera is mounted on said back end of said mobile agent, said camera is tilted to have a field of view of said ground path direct behind said mobile agent.

16. A non-transitory computer-readable medium encoded with a computer program code for causing a processor to execute a method for adding a waypoint to an ordered plurality of waypoints, said method comprising:
receiving a new waypoint;
adding said new waypoint to a terminal end of an ordered plurality of waypoints associated with a mobile-agent path;
determining if a mobile agent is running; and
when said mobile agent is not running:
updating a target waypoint to said new waypoint; and
driving toward said target waypoint.

17. A non-transitory computer-readable medium encoded with a computer program code for causing a processor to correct for drift, said method comprising:
computing a current position and a current heading of a mobile agent;
determining if said mobile agent is close to a target waypoint; and
when said mobile agent is determined to be close to said target waypoint:
determining if said target waypoint is associated with a machine-readable code; and
when said target waypoint is determined to be associated with a machine-readable code:
identifying said machine-readable code;
computing a position of said mobile agent relative to said machine-readable code;
determining an actual position and

Assignees

• Sharp Laboratories America Inc

Inventors

• Liao Miao

Classifications

• G01C21/20 Primary

  Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) · CPC title

• B60R11/04 Primary

  Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T7/73

  using feature-based methods · CPC title