Gathering data in a communication system

Patent metadata

Field	Value
Publication number	US-11710080-B2
Application number	US-201916443734-A
Country	US
Kind code	B2
Filing date	Jun 17, 2019
Priority date	Sep 27, 2018
Publication date	Jul 25, 2023
Grant date	Jul 25, 2023

Abstract

A computer-implemented method comprising: outputting questions to a user via one or more user devices, and receiving back responses to some of the questions from the user via one or more user devices; over time, controlling the outputting of the questions so as to output the questions under circumstances of different values for each of one or more items of metadata, wherein the one or more items of metadata comprise at least a time and/or a location at which a question was output to the user via the one or more user devices; monitoring whether or not the user responds when the question is output with the different metadata values; training the machine learning algorithm to learn a value of each of the items of metadata which optimizes a reward function, and based thereon selecting a time and/or location at which to output subsequent questions.

First claim

The invention claimed is:

1. Computing apparatus comprising a processor and storage storing code arranged to run on the processor, wherein the code is configured so as when run to perform operations of:
outputting questions to a user via a user device, and receiving back responses to some of the questions from the user via the user device;
over time, controlling the outputting of the questions so as to output the questions with a different value associated with each item of a plurality of items of metadata, wherein the plurality of items of metadata comprise at least one of a time or a location at which a question was output to the user via the user device;
determining a probability distribution of an unanswered question of the questions; and
training a machine learning algorithm to learn a value of each of the items of metadata which optimizes a reward function based on the determined probability distribution, and based thereon selecting at least one of a time or location at which to output a subsequent question, wherein the reward function comprises a distance between the probability distribution of the unanswered question and a probability distribution based on an answered question of the questions.

2. The apparatus according to claim 1 , wherein said machine learning algorithm comprises a first neural network and a second neural network, wherein said training comprises:
inputting a vector comprising input values to the first neural network in order to generate a respective set of first values from each set of input values, each set of first values representing a probability distribution of a respective one of the input values, and each input value corresponding to a different respective combination of question and associated value of the metadata, wherein each input value represents either the user's response to the respective question under circumstances of the respective metadata or that the user did not respond to the respective question under circumstances of the respective metadata;
inputting some or all of the sets of first values to the second neural network in order to generate a respective set of second values from each first set input to the second neural network, each set of second values representing a probability distribution of a respective combination of a predicted response to the respective question under circumstances of its associated metadata; and
supplying the sets of second values to the reward function to perform said optimization, wherein said reward function comprises an expectation of a distance between (a) each respective probability distribution of the predicted responses and associated metadata of the question not responded to by the user, and (b) an overall probability distribution combining the predicted responses to questions and associated metadata of the questions that have been responded to by the user,
wherein said optimization comprises, for a subsequent question to be output to the user, selecting the metadata value that maximizes said distance between a probability distribution of the predicted response to a question with that metadata value.

3. The apparatus according to claim 2 , wherein said selecting comprises selecting both which question is to be the subsequent question and the value of its associated metadata, by selecting a combination of subsequent question and associated metadata value that maximizes said distance.

4. The apparatus according to claim 2 , wherein each input value of the vector input to first neural network is a respective sub-vector of that vector, wherein each sub-vector is generated by inputting the respective input value and a respective identifier of the respective question into a third neural network that performs a symmetric operation on the respective combination of input value and question identifier.

5. The apparatus according to claim 1 , wherein said selecting the at least one of the time or the location at which to output the subsequent question comprises selecting at least one of the time or the location at which to output the question that maximises the reward function.

6. The apparatus according to claim 1 , wherein each question is associated with an estimate of cost to the user, and wherein a next one of the subsequent question selected to be output to the user is the question that maximises an amount of information content gained per unit cost to the user.

7. The apparatus according to claim 1 , wherein training the machine learning algorithm comprises, for a subsequent question to be output to the user, selecting the value that maximizes the distance between the probability distribution of the unanswered question and the probability distribution based on the answered question of the questions.

8. The apparatus according to claim 1 , wherein the reward function comprises a measure of responsivity of the user, and wherein optimizing the reward function comprises increasing the measure of responsivity of the user.

9. The apparatus according to claim 8 , wherein the measure of responsivity of the user comprises at least one of: (a) a number of responses from the user per unit time, (b) a number of responses per question asked, and (c) an engagement with the question from the user.

10. The apparatus according to claim 1 , wherein the code is further configured to supply the responses to a scoring algorithm to generate scores predicting a condition of the user based on the responses.

11. The apparatus according to claim 10 , wherein the reward function comprises a measure of prediction quality, and wherein said optimization comprises optimizing a trade-off between a measure of responsivity and the prediction quality based on the generated scores.

12. The apparatus according to claim 11 , wherein the measure of prediction quality comprises at least one of:
a statistical uncertainty or variability in the scores, or
a comparison with subsequently obtained empirical information on the condition of the user.

13. The apparatus according to claim 10 , wherein the questions relate to a health condition of the user, and the predicted condition comprises the health condition.

14. The apparatus according to claim 1 , wherein the questions output under circumstances of the different values of a given one of the items of metadata comprise:
some repeated instances of a same question, and some different questions.

15. The apparatus according to claim 1 , wherein the items of metadata comprise contextual information reflecting a context in which the questions are output, wherein the contextual information comprises one or more of:
a pattern of motion of the user when the questions are asked,
an activity being conducted by the user when the questions are asked,
a cause which prompted the question to be asked,
a sleep pattern of the user around the times the questions are asked,
a facial expression of the user when the questions are asked,
a change in heart rate when the questions are asked,
a change in cadence of speech when responding through a voice interface,
a change in intonation when responding through a voice interface,
a change in cadence of typing when responding in a chat-bot user interface,
a change in sentiment when responding in a chat-bot user interface.

16. The apparatus according to claim 1 , wherein the items of metadata comprise a controllable parameter of the questions or a manner in which the questions are output, wherein the controllable parameter comprises one or more of:
a frequency at which the questions are asked,
how many questions are asked per sitting,
which user device is used to as

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Zhang Cheng
• Nowozin Reinhard Sebastian Bernhard
• Patel Ameera
• Belgrave Danielle Charlotte Mary
• Palla Konstantina
• Thieme Anja
• Buchan Iain Edward
• Ma Chao
• Tschiatschek Sebastian
• Hernandez Lobato Jose Miguel

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/091

  Active learning · CPC title