Autonomous human-centric place recognition

Patent metadata

Field	Value
Publication number	US-10049267-B2
Application number	US-201615057032-A
Country	US
Kind code	B2
Filing date	Feb 29, 2016
Priority date	Feb 29, 2016
Publication date	Aug 14, 2018
Grant date	Aug 14, 2018

Abstract

The novel technology described in this disclosure includes an example method comprising capturing sensor data using one or more sensors describing a particular environment; processing the sensor data using one or more computing devices coupled to the one or more sensors to detect a participant within the environment; determining a location of the participant within the environment; querying a feature database populated with a multiplicity of features extracted from the environment using the location of the participant for one or more features being located proximate the location of the participant; and selecting, using the one or more computing devices, a scene type from among a plurality of predetermined scene types based on association likelihood values describing probabilities of each feature of the one or more features being located within the scene types.

First claim

What is claimed is:

1. A computer-implemented method comprising:
capturing sensor data using one or more sensors describing a particular environment;
processing the sensor data using one or more computing devices coupled to the one or more sensors to detect a participant within the particular environment;
determining a location of the participant within the particular environment using depth image data, the depth image data describing at least a depth of points in the particular environment relative to the one or more sensors;
querying a feature database populated with a multiplicity of features extracted from the particular environment using the location of the participant for one or more features located within a search area defined by a search area dimension relative to the location of the participant, the one or more features representing one or more physical objects in the particular environment, the search area dimension including at least the depth based on the depth image data; and
selecting, using the one or more computing devices, a scene type from among a plurality of predetermined scene types based on association likelihood values describing probabilities of each feature, of the one or more features located within the search area defined by the search area dimension relative to the location of the participant, being located within the scene type selected from among the plurality of predetermined scene types.

2. The computer-implemented method of claim 1 , further comprising:
executing one or more autonomous routines based on the selected scene type.

3. The computer-implemented method of claim 1 , further comprising:
generating the association likelihood values based on times of day.

4. The computer-implemented method of claim 1 , wherein the association likelihood values are classification scores respectively describing the probabilities of each feature being located within the scene type, prior probabilities of classifying the scene types correctly using the features, N-dimensional locations with respect to an arbitrary reference point, and sizes.

5. The computer-implemented method of claim 1 , wherein the association likelihood values are probabilities computed using a combined scene probability for each scene type.

6. The computer-implemented method of claim 1 , further comprising:
prior to querying the feature database, scanning the particular environment using the one or more sensors;
extracting the multiplicity of features of the particular environment using sensor data provided by the one or more sensors responsive to scanning the particular environment; and
populating the feature database with the multiplicity of features.

7. The computer-implemented method of claim 1 , wherein selecting the scene type includes generating a gradient for an area in a vicinity of the participant, determining a directionality based on the gradient, and selecting the scene type further based on the directionality of the gradient.

8. The computer-implemented method of claim 1 , wherein at least one of the one or more sensors is an RGB-D camera.

9. The computer-implemented method of claim 1 , wherein the participant is a human.

10. An autonomous computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
capturing sensor data using one or more sensors describing a particular environment,
processing the sensor data using one or more computing devices coupled to the one or more sensors to detect a participant within the particular environment,
determining a location of the participant within the particular environment using depth image data, the depth image data describing at least a depth of points in the particular environment relative to the one or more sensors,
querying a feature database populated with a multiplicity of features extracted from the particular environment using the location of the participant for one or more features located within a search area defined by a search area dimension relative to the location of the participant, the one or more features representing one or more physical objects in the particular environment, the search area dimension including at least the depth based on the depth image data, and
selecting, using the one or more computing devices, a scene type from among a plurality of predetermined scene types based on association likelihood values describing probabilities of each feature, of the one or more features located within the search area defined by the search area dimension relative to the location of the participant, being located within the scene type of the plurality of predetermined scene types.

11. The system of claim 10 , wherein the instructions, when executed by the one or more processors, further cause the system to perform operations comprising:
executing one or more autonomous routines based on the selected scene type.

12. The system of claim 10 , wherein the instructions, when executed by the one or more processors, further cause the system to perform operations comprising:
generating the association likelihood values.

13. The system of claim 10 , wherein the association likelihood values are classification scores respectively describing the probabilities of each feature being located within the scene type, prior probabilities of classifying the scene types correctly using the features, N-dimensional locations with respect to an arbitrary reference point, and sizes.

14. The system of claim 10 , wherein the association likelihood values are probabilities computed using a combined scene probability for each scene type.

15. The system of claim 10 , wherein the instructions, when executed by the one or more processors, further cause the system to perform operations comprising:
prior to querying the feature database, scanning the particular environment using the one or more sensors,
extracting the multiplicity of features of the particular environment using sensor data provided by the one or more sensors responsive to scanning the particular environment, and populating the feature database with the multiplicity of features.

16. The system of claim 10 , wherein selecting the scene type includes generating a gradient for an area in a vicinity of the participant, determining a directionality based on the gradient, and selecting the scene type further based on the directionality of the gradient.

17. The system of claim 10 , wherein at least one of the one or more sensors is an RGB-D camera.

18. The system of claim 10 , wherein the participant is a human.

19. The computer-implemented method of claim 1 , wherein the search area dimension relative to the location of the participant includes a defined distance from the location of the participant.

20. The system of claim 10 , wherein the search area dimension relative to the location of the participant includes a defined distance from the location of the participant.

Assignees

• Toyota Motor Co Ltd

Inventors

• Martinson Eric
• Kim David
• Nakano Yusuke

Classifications

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06F16/24575 Primary

  using context · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N20/00

  Machine learning · CPC title