Methods and apparatus for autonomous robotic control

Patent metadata

Field	Value
Publication number	US-9626566-B2
Application number	US-201514662657-A
Country	US
Kind code	B2
Filing date	Mar 19, 2015
Priority date	Mar 19, 2014
Publication date	Apr 18, 2017
Grant date	Apr 18, 2017

Abstract

Sensory processing of visual, auditory, and other sensor information (e.g., visual imagery, LIDAR, RADAR) is conventionally based on “stovepiped,” or isolated processing, with little interactions between modules. Biological systems, on the other hand, fuse multi-sensory information to identify nearby objects of interest more quickly, more efficiently, and with higher signal-to-noise ratios. Similarly, examples of the OpenSense technology disclosed herein use neurally inspired processing to identify and locate objects in a robot's environment. This enables the robot to navigate its environment more quickly and with lower computational and power requirements.

First claim

The invention claimed is:

1. A system comprising:
an image sensor to acquire a plurality of images of at least a portion of an environment surrounding a robot; and
a processor, operably coupled to the image sensor, to:
translate each image in the plurality of images from a frame of reference of the image sensor to an allocentric frame of reference;
identify a position, in the allocentric frame of reference, of an object appearing in at least one image in the plurality of images; and
determine if the object appears in at least one other image in the plurality of images based on the position, in the allocentric frame of reference, of the object.

2. The system of claim 1 , wherein the processor is configured to translate the at least one image from the frame of reference of the image sensor to an allocentric frame of reference by:
translating each image in the plurality of images from the frame of reference of the image sensor to an egocentric frame of reference based on a position and/or an orientation of the image sensor in the egocentric frame of reference, the egocentric frame of reference being defined with respect to the robot; and
translating each image in the plurality of images from the egocentric frame of reference to the allocentric frame of reference.

3. The system of claim 1 , wherein the processor is configured to identify the position in the allocentric frame of reference of the object by:
generating a segmented version of the at least one image in the plurality of images; and
determining at least one spatial shroud fitting a form of the object based at least in part on the segmented version of the at least one image.

4. The system of claim 3 , wherein the processor is configured to determine if the object appears in at least one other image in the plurality of images at least in part on by:
applying the at least one spatial shroud to the other image in the plurality of images.

5. The system of claim 1 , wherein the processor is configured to:
map the position, in the allocentric frame of reference, of the object to coordinates in the frame of reference of the image sensor; and
determine a change to a position and/or an orientation of the image sensor based at least in part on the coordinates in the frame of reference of the image sensor.

6. The system of claim 5 , further comprising:
an actuator, operably coupled to the processor and to the image sensor, to adjust a field of view of the image sensor based at least in part on the change to the position and/or the orientation of the image sensor, and
wherein the image sensor is configured to acquire a subsequent image in the plurality of images in response to adjustment of the field of view.

7. A method of locating an object with respect to a robot, the method comprising:
(A) acquiring, with a image sensor coupled to the robot, a plurality of images of at least a portion of an environment surrounding the robot;
(B) automatically translating each image in the plurality of images from a frame of reference of the image sensor to an allocentric frame of reference;
(C) identifying a position, in the allocentric frame of reference, of an object appearing in at least one image in the plurality of images; and
(D) determining if the object appears in at least one other image in the plurality of images based on the position, in the allocentric frame of reference, of the object.

8. The method of claim 7 , wherein (B) comprises:
(B1) translating each image in the plurality of images from the frame of reference of the image sensor to an egocentric frame of reference based on a position and/or an orientation of the image sensor in the egocentric frame of reference, the egocentric frame of reference being defined with respect to the robot; and
(B2) translating each image in the plurality of images from the egocentric frame of reference to the allocentric frame of reference.

9. The method of claim 7 , wherein (C) comprises:
(C1) generating a segmented version of a first image in the plurality of images; and
(C2) determining a spatial shroud fitting a form of the object based at least in part on the segmented version of the first image.

10. The method of claim 9 , further comprising:
(E) mapping the position, in the allocentric frame of reference, of the object to coordinates in the frame of reference of the image sensor; and
(F) determining a change to a position and/or an orientation of the image sensor based at least in part on the coordinates in the frame of reference of the image sensor.

11. The method of claim 10 , wherein (F) further comprises positioning and/or orienting the image sensor away from the object.

12. The method of claim 10 , wherein (F) further comprises positioning and/or orienting the image sensor to acquire another image of the object.

13. The method of claim 10 , wherein:
(D) comprises translating and/or transforming the spatial shroud based at least in part on the change in the position and/or the orientation of the image sensor determined in (F), and
(C) comprises determining if the spatial shroud fits the form of the object in a segmented version of a second image in the plurality of images.

14. The method of claim 13 , wherein (D) further comprises:
(D3) identifying at least one feature of the object in the first image; and
(D4) comparing the at least one feature to a plurality of features identified in other images in the plurality of images.

15. The method of claim 10 , further comprising:
(G) adjusting a field of view of the image sensor based at least in part on the change to the position and/or the orientation of the image sensor.

16. The method of claim 15 , wherein (A) comprises:
acquiring a subsequent image in the plurality of images in response to adjustment of the field of view.

Assignees

• Neurala Inc
• Univ Boston

Inventors

• Versace Massimiliano
• Gorshechnikov Anatoly
• Livitz Gennady
• Palma Jesse

Classifications

• G06N3/049 Primary

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/30244

  Camera pose · CPC title