System and method for gesture-based point of interest search

Patent metadata

Field	Value
Publication number	US-11275447-B2
Application number	US-201816058704-A
Country	US
Kind code	B2
Filing date	Aug 8, 2018
Priority date	Mar 15, 2013
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

A user, such as the driver of a vehicle, to retrieve information related to a point of interest (POI) near the vehicle by pointing at the POI or performing some other gesture to identify the POI. Gesture recognition is performed on the gesture to generate a target region that includes the POI that the user identified. After generating the target region, information about the POI can be retrieved by querying a server-based POI service with the target region or by searching in a micromap that is stored locally. The retrieved POI information can then be provided to the user via a display and/or speaker in the vehicle. This process beneficially allows a user to rapidly identify and retrieve information about a POI near the vehicle without having to navigate a user interface by manipulating a touchscreen or physical buttons.

First claim

What is claimed is:

1. A vehicle-based computer-implemented method for retrieving information associated with a point of interest, the method comprising:
accessing, at a computer system, electronic movement data representing a current speed of a vehicle and electronic location data representing a current location and an orientation of the vehicle based on an electronic reception of electronic sensor data;
analyzing the electronic movement data and the electronic location data to determine a region that the vehicle is likely to travel through;
receiving an electronic data signal representing a gesture performed by a user inside the vehicle;
performing gesture recognition on the electronic data signal to determine a direction vector representing a direction the gesture indicates, wherein the direction vector is a multi-dimensional vector that represents a direction of the gesture performed by the user relative to a pair of axes that are parallel to a direction of travel of the vehicle and perpendicular to the direction of travel of the vehicle;
receiving a voice command issued by the user inside the vehicle;
performing electronic voice recognition on the voice command to electronically determine one or more words spoken by the user as part of the voice command;
analyzing, at the computer system, the direction vector, the electronic movement data, the electronic location data, the region that the vehicle is likely to travel through and the words spoken by the user to identify a point of interest (POI) that is of interest to the user that aligns with the direction vector, wherein a point of interest search is performed by electronically accessing and electronically querying an electronic datastore to determine information associated with the POI; and
executing electronic instructions to electronically control a display device to provide the information associated with the POI.

2. The computer-implemented method of claim 1 , further including generating a target region corresponding to the direction vector, the electronic movement data, and the electronic location data;
wherein identifying the POI includes determining that the POI is inside the target region, wherein the target region aligns with the direction vector and is dependent on the current speed of the vehicle.

3. The computer-implemented method of claim 2 , wherein a size of the target region increases as a speed of the vehicle increases.

4. The computer-implemented method of claim 2 , wherein:
depth information within the data signal is utilized to generate a three-dimensional depth reconstruction of the gesture that pertains to the target region, wherein
the location data includes an altitude of the vehicle, and
the target region is three-dimensional based on the three-dimensional depth reconstruction of the gesture.

5. The computer-implemented method of claim 2 , further including:
accessing from a local storage, information associated with one or more points of interest that are located in the region that the vehicle is likely to travel through, wherein the one or more points of interest include the POI; and
determining locations of the one or more points of interest relative to the target region;
wherein the POI is identified from among the one or more points of interest by determining that the POI is inside the target region.

6. The computer-implemented method of claim 5 , wherein if the information associated with the POI is not in the local storage, then the method further includes:
querying a remote server with the target region to identify the POI, and
accessing the information associated with the POI from the remote server.

7. The computer-implemented method of claim 6 , wherein if two or more points of interest, including the POI, are inside the target region and are identified as being of interest to the user, then the method further includes using the one or more words spoken by the user to identify the POI from among the two or more points of interest.

8. The computer-implemented method of claim 6 , wherein if two or more points of interest, including the POI, are inside the target region and are identified as being of interest to the user, then the method further includes iteratively decreasing a size of the target region until only the POI is inside the target region.

9. The computer-implemented method of claim 6 , wherein:
if two or more points of interest, including the POI, are inside the target region and are identified as being of interest to the user, then the two or more points of interest, including the POI, is provided to the user in a list; and
the information associated with the POI is provided to the user upon the POI being selected from the list by the user.

10. The computer-implemented method of claim 6 , wherein if zero points of interest are inside the target region, then the method further includes iteratively increasing a size of the target region until only the POI is inside the target region.

11. A non-transitory computer-readable storage medium for storing computer program instructions for retrieving information associated with a point of interest (POI), the program instructions when executed by a processor cause the processor to perform steps including:
accessing electronic movement data representing a current speed of a vehicle and electronic location data representing a current location and an orientation of the vehicle based on an electronic reception of electronic sensor data;
analyzing the electronic movement data and the electronic location data to determine a region that the vehicle is likely to travel through;
receiving an electronic data signal representing a gesture performed by a user inside the vehicle;
performing gesture recognition on the electronic data signal to determine a direction vector representing a direction the gesture indicates, wherein the direction vector is a multi-dimensional vector that represents a direction of the gesture performed by the user relative to a pair of axes that are parallel to a direction of travel of the vehicle and perpendicular to the direction of travel of the vehicle;
receiving a voice command issued by the user inside the vehicle;
performing electronic voice recognition on the voice command to electronically determine one or more words spoken by the user as part of the voice command;
analyzing, at the computer system, the direction vector, the electronic movement data, the electronic location data, the region that the vehicle is likely to travel through and the words spoken by the user to identify a POI that is of interest to the user that aligns with the direction vector, wherein a point of interest search is performed by electronically accessing and electronically querying an electronic datastore to determine information associated with the POI; and
executing electronic instructions to electronically control a display device to provide the information associated with the POI.

12. The non-transitory computer readable storage medium of claim 11 , wherein the program instructions further cause the processor to perform steps including generating a target region corresponding to the direction vector, the electronic movement data, and the electronic location data;
wherein identifying the POI includes determining that the POI is inside the target region, wherein the target region aligns with the direction vector and is dependent on the current speed of the vehicle.

13. The non-transitory computer readable storage medium of claim 12 , wherein a size of the target region increases as a speed of the vehicle increases.

14. The non-transitory computer readable storage med

Assignees

• Honda Motor Co Ltd
• Edge 3 Tech Llc

Inventors

• Vaghefinazari Pedram
• Yamamoto Stuart Masakazu
• El Dokor Tarek A
• Cluster Jordan
• Holmes James E

Classifications

• B60K35/60

  Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10) · CPC title

• B60K35/26

  using acoustic output · CPC title

• B60K35/85

  Arrangements for transferring vehicle- or driver-related data · CPC title

• B60K35/28

  characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver · CPC title

• B60K35/22

  Display screens · CPC title