Interactive search experience using machine learning

Patent metadata

Field	Value
Publication number	US-2020341976-A1
Application number	US-201916394853-A
Country	US
Kind code	A1
Filing date	Apr 25, 2019
Priority date	Apr 25, 2019
Publication date	Oct 29, 2020
Grant date	—

Abstract

Techniques are disclosed for providing an interactive search session. The interactive search session is implemented using an artificial intelligence model. For example, when the artificial intelligence model receives a search query from a user, the model selects an action from a plurality of actions based on the search query. The selected action queries the user for more contextual cues about the search query (e.g., may enquire about use of the search results, may request to refine the search query, or otherwise engage the user in conversation to better understand the intent of the search). The interactive search session may be in the form, for example, of a chat session between the user and the system, and the chat session may be displayed along with the search results (e.g., in a separate section of display). The interactive search session may enable the system to better understand the user's search needs, and accordingly may help provide more focused search results.

First claim

What is claimed is:

1 . A method for providing an interactive search session, the method comprising:
receiving, at a search engine, a search query from a device, the search query provided by a user, the search engine configured with a Reinforcement Learning (RL)-based agent programmed to interact with the user, to help the user in refining the search query by providing the user with contextual assistance; generating, by the RL-based agent and based at least in part on the search query, a vector representation of a plurality of values corresponding to a plurality of possible actions that the search engine can take in response to the search query, each of the possible actions associated with a corresponding one of the values that indicates a rank of that action relative to the other possible actions, wherein a given value encodes a sequential aggregation of one or both agent and user actions in last k cycles of the search session to capture both a local context and a global context, wherein one cycle of the search session includes a first action by the user and a second action by the search engine, and the local context includes a current cycle and/or a just previous cycle, and the global context includes one or more relatively older historical cycles not reflected in the local context; selecting, by the RL-based agent, an action from the possible actions, based at least in part on the value corresponding to the selected action; transmitting, by the search engine, a message to the device, for displaying on the device, the message based at least in part on the selected action, wherein the message is different from results of the search query and solicits further action from the user; and refining the search query based at least in part on action by the user that is responsive to the message.

2 . The method of claim 1 , wherein the selected action is a first selected action, the method further comprising:
receiving, by the search engine, a user response to the message from the device; generating, by the RL agent and based at least in part on the user response, another vector representation of a plurality of values corresponding to a plurality of possible actions that the search engine can take in response to the user response, each of the actions associated with a corresponding one of said values that reflects one or both agent and user actions in the most recent cycle of the search session; selecting, by the RL agent, a second action from the possible actions, based at least in part on the value corresponding to the second selected action; and transmitting, by the search engine, another message to the device, based at least in part on the second selected action.

3 . The method of claim 2 , further comprising:
iteratively repeating receiving a user response, generating another vector, selecting a corresponding action from the possible actions, and transmitting a corresponding message to the device, until an end of the search session is identified.

4 . The method of claim 1 , wherein the selected action and the message are to request additional information about the search query, or to request to refine the search query.

5 . The method of claim 1 , further comprising:
retrieving, by the search engine and from one or more databases, search results responsive to the search query; and providing, by the search engine, the search results to the device to cause display of the search results on the device simultaneously with a display of the message on the device.

6 . The method of claim 1 , wherein the RL-based agent comprises an artificial intelligence model that is trained artificially through a virtual user.

7 . The method of claim 1 , wherein the RL-based agent operates on an Asynchronous Advantage Actor-Critic (A3C) algorithm that generates an actor output and a critic output, the actor output including the possible actions and the critic output including a state of the search session, the state encoding the agent actions and user actions in the last k cycles of the search session.

8 . The method of claim 1 , further comprising:
training the RL-based agent by causing the RL-based agent to interact with a virtual user, wherein the virtual user is modelled using conversation history of the RL-based agent with one or more actual users.

9 . The method of claim 1 , further comprising:
training the RL-based agent using rewards, wherein rewards awarded during a search episode include
a first reward for successful completion of the search episode,
a second reward, based on user response to an action selected by the RL-based agent, and
a third reward for completion of an auxiliary task identified by the RL-based agent,
wherein a training episode includes a sequence of interactions between a virtual user and the RL-based agent.

10 . The method of claim 1 , further comprising:
iteratively repeating generating a vector and selecting a corresponding action; and defining, at each cycle of the search session, a corresponding state of the RL-based agent, wherein the state at a specific cycle includes one or more of
at least a partial history of actions selected so far in the search session,
at least a partial history of responses received from the device so far in the search session, and/or
a length of the search session so far.

11 . The method of claim 10 , wherein at least the partial history of actions selected so far in the search session comprises a plurality of action vectors, wherein each action vector is indicative of a corresponding action undertaken during a corresponding cycle of the search session.

12 . The method of claim 10 , wherein at least the partial history of responses received from the device so far in the search session comprises a plurality of user vectors, wherein each user vector is indicative of a corresponding response received during a corresponding cycle of the search session.

13 . A system comprising:
one or more processors; a processing module executable by the one or more processors to receive a user message from a device and to parse the user message, the user message comprising at least one of: a search query, or a response from a user during an interaction of the user with the system, the interaction is to generate results of the search query; a search engine to receive the parsed user message, and to generate search results based on the parsed user message; and an artificial intelligence module executable by the one or more processors to
receive the parsed user message and the search results,
generate a vector representation of a plurality of values corresponding to a plurality of possible actions that the system can take, based at least in part on the parsed user message and the search results, each of the possible actions associated with a corresponding one of the values that indicates a rank of that action relative to the other possible actions, and
select a first action of the possible actions, based at least in part on a value corresponding to the selected action.

14 . The system of claim 13 , wherein the processing module is a natural language processing (NLP) module, and wherein processing module is to:
receive an indication of the selection of the first action of the possible actions; generate a system message, based on the indication of the selection of the first action; and cause the system message to be transmitted to the device, for displaying on the device.

15 . The system of claim 14 , wherein the system message includes at least one of the search results, a request for more information about the search query, a r

Assignees

• Adobe Inc

Inventors

• Aggarwal Milan
• Krishnamurthy Balaji

Classifications

• G06F16/243 Primary

  Natural language query formulation · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06F16/248

  Presentation of query results · CPC title