System and method for creating and sharing plans through multimodal dialog

Patent metadata

Field	Value
Publication number	US-10739976-B2
Application number	US-201815872419-A
Country	US
Kind code	B2
Filing date	Jan 16, 2018
Priority date	Dec 19, 2014
Publication date	Aug 11, 2020
Grant date	Aug 11, 2020

Abstract

Methods, systems, devices, and media for creating a plan through multimodal search inputs are provided. A first search request comprises a first input received via a first input mode and a second input received via a different second input mode. The second input identifies a geographic area. First search results are displayed based on the first search request and corresponding to the geographic area. Each of the first search results is associated with a geographic location. A selection of one of the first search results is received and added to a plan. A second search request is received after the selection, and second search results are displayed in response to the second search request. The second search results are based on the second search request and correspond to the geographic location of the selected one of the first search results.

First claim

What is claimed is:

1. A method of creating a plan through search inputs, the method comprising:
receiving a first search request by a device, the first search request comprising a first input received by the device via a first input mode and a second input received by the device via a second input mode different than the first input mode, the second input identifying a geographic area;
displaying first search results on a display of the device in response to receiving the first search request, the first search results being based on the first search request and corresponding to the geographic area, each of the first search results being associated with a geographic location;
receiving a selection of one of the first search results by the device;
adding the one of the first search results to a plan;
receiving a second search request by the device after receiving the selection; and
displaying second search results on the display of the device in response to receiving the second search request, the second search results being based on the second search request and corresponding to the geographic location of the one of the first search results,
wherein the second input includes a movement of the device, and
the geographic area is identified based on the movement of the device.

2. The method according to claim 1 , wherein
the movement of the device includes the device being moved in a particular direction, and
the geographic area is identified as being in the particular direction from a current position of the device.

3. The method according to claim 1 , wherein
the movement of the device includes the device being aimed in a particular direction, and
the geographic area is identified as being toward the particular direction relative to the device.

4. The method according to claim 1 , wherein
the movement of the device includes the device being positioned relative to a predetermined object, and
the geographic area is specified based on a relative position of the device to the predetermined object.

5. The method according to claim 1 , wherein
the first input comprises a spoken query that is received by a microphone of the device, the spoken query specifying a type of the first search results.

6. The method according to claim 5 , wherein
the second input further includes a touch query that is received by the device via a touch operation to the device, and
the geographic area is identified further based on the touch query that is received by the device.

7. The method according to claim 6 , wherein
the geographic area is identified based on the movement of the device, and
the geographic area is refined based on the touch query.

8. The method according to claim 7 , further comprising:
displaying, on the display of the device, a map of the geographic area that is identified based on the movement of the device, the first search results being displayed in the map,
wherein the geographic area is refined based on the touch query by correcting the geographic area displayed on the map, narrowing the geographic area displayed on the map, or broadening the geographic area displayed on the map.

9. The method according to claim 8 , wherein
the geographic area is refined by at least one of a portion of the geographic area being encircled on the map, a portion of the geographic area being contacted on the map, a zoom-in operation being received by the device, or a zoom-out operation being received by the device.

10. The method according to claim 1 , wherein
the second input includes an image being received by the device, and
the geographic area is further identified based on the image.

11. The method according to claim 10 , wherein
the image is captured by a camera of the device, and
the captured image is analyzed to identify a location within the image for identifying the geographic area.

12. The method according to claim 10 , wherein
the image is downloaded to the device from the Internet or obtained by the device from a friend, and
the downloaded or obtained image is analyzed to identify a location within the image for identifying the geographic area.

13. The method according to claim 10 , wherein
the first input comprises a spoken query that is received by a microphone of the device, the spoken query specifying a type of the first search results.

14. The method according to claim 13 , wherein
the second input further includes a touch query that is received by the device via a touch operation to the device, and
the geographic area is identified further based on the touch query that is received by the device.

15. The method according to claim 14 , wherein
the geographic area is identified based on the image received by the device, and
the geographic area is refined based on the touch query.

16. The method according to claim 15 , further comprising:
displaying, on the display of the device, a map of the geographic area that is identified based on the image received by the device, the first search results being displayed in the map,
wherein the geographic area is refined based on the touch query by correcting the geographic area displayed on the map, narrowing the geographic area displayed on the map, or broadening the geographic area displayed on the map.

17. The method according to claim 1 , further comprising:
displaying, on the display of the device, the second search results on a map in correspondence with the geographic location of the one of the first search results;
receiving a request to modify the second search results displayed on the display by the device, the request including a refinement of a displayed area of the map on the display;
determining whether the request to modify the second search results includes one of the refinement of the displayed area and a correction of a type of the second search results; and
modifying the second search results on the display of the device based on the refinement of the displayed area.

18. A device for creating a plan through search inputs, the device comprising:
a display;
a processor; and
a memory storing instructions that, when executed by the processor, cause the processor to perform operations including:
receiving a first search request, the first search request comprising a first input received by the device via a first input mode and a second input received by the device via a second input mode different than the first input mode, the second input identifying a geographic area;
displaying first search results on the display in response to receiving the first search request, the first search results being based on the first search request and corresponding to the geographic area, each of the first search results being associated with a geographic location;
receiving a selection of one of the first search results;
adding the one of the first search results to a plan;
receiving a second search request after receiving the selection; and
displaying second search results on the display in response to receiving the second search request, the second search results being based on the second search request and corresponding to the geographic location of the one of the first search results,

wherein the second input includes a movement of the device, and
the geographic area is identified based on the movement of the device.

19. A tangible computer-readable medium encoded with an executable computer program for creating a plan through search inputs that, when executed by a processor, causes the processor to perform operations

Assignees

• At & T Ip I Lp

Inventors

• Johnston Michael J
• Ehlen Patrick
• Jung Hyuckchul
• Lieske Jr Jay H
• Selfridge Ethan
• Vasilieff Brant J
• Wilpon Jay Gordon

Classifications

• G06F16/9537 Primary

  Spatial or temporal dependent retrieval, e.g. spatiotemporal queries · CPC title

• G06F3/04847 Primary

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title