Providing emotional care in a session

Patent metadata

Field	Value
Publication number	US-11810337-B2
Application number	US-202217752149-A
Country	US
Kind code	B2
Filing date	May 24, 2022
Priority date	Jan 4, 2018
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

A method and apparatus for providing emotional care in a session between a user and conversational agent. A first group of images comprising one or more images associated with the user may be received in the session. A user profile may be obtained. A first group of textual descriptions may be generated from the first group of images based at least on emotion information in the user profile. A first memory record may be created based at least on the first group of images and the first group of textual descriptions. A second group of images may be received in the session to generate a second group of textual descriptions from the second group of images based at least on the emotion information in the user profile. A second memory record may be created based at least on the second group of images and the second group of textual descriptions.

First claim

What is claimed is:

1. A method for providing emotional care in a session between a user and an electronic conversational agent, comprising:
receiving a first group of images in the session, the first group of images comprising one or more images associated with the user;
obtaining a user profile of the user;
generating a first group of textual descriptions from the first group of images based at least on emotion information in the user profile;
creating a first memory record based at least on the first group of images and the first group of textual descriptions;
receiving a second group of images in the session;
generating a second group of textual descriptions from the second group of images based at least on the emotion information in the user profile; and
creating a second memory record based at least on the second group of images and the second group of textual descriptions.

2. The method of claim 1 , further comprising:
generating a question for inquiring relationship between the first memory record and the second memory record;
providing the question in the session;
receiving an answer indicating the relationship in the session; and
creating a joint memory record based at least on the first memory record, the second memory record and the relationship.

3. The method of claim 1 , further comprising:
providing a response in the session based on the first memory record.

4. The method of claim 1 , further comprising:
storing the first memory record in a memory record database, the memory record database including a plurality of memory records.

5. The method of claim 4 , further comprising:
receiving a message in the session;
retrieving a memory record associated with the message from the memory record database; and
providing a response in the session based on the retrieved memory record.

6. The method of claim 1 , wherein each of the first group of textual descriptions is generated through neural networks in a word-by-word approach.

7. The method of claim 6 , wherein during generating a textual description, a next word of the textual description is generated based on at least one of: emotion information in the user profile that is associated with a generated part of the textual description; and emotion information obtained through performing emotion analysis on the generated part of the textual description.

8. The method of claim 7 , wherein the emotion information in the user profile comprises a plurality of emotion information items, and each emotion information item comprises at least one of: a topic of interest, opinion on the topic, and emotion category of the opinion.

9. The method of claim 6 , wherein during generating a textual description, a generated part of the textual description is semantically extended by at least one of: fact information in the user profile, and knowledge information in a knowledge graph.

10. The method of claim 9 , wherein the fact information in the user profile comprises a plurality of fact information items, and each fact information item comprises at least one of: a topic of interest, action, location and time.

11. The method of claim 1 , further comprising:
performing a psychological test or a cognitive test in the session.

12. The method claim 11 , wherein each of questions in the psychological test or the cognitive test is determined based on at least one of: an answer from the user to a previous question, and occurrence frequencies of topics in the user profile.

13. The method claim 12 , wherein each of questions in the psychological test or the cognitive test is determined based on mutex criteria.

14. The method claim 11 , wherein a test result of the psychological test or cognitive test is determined based on answers from the user and/or answers from at least one another user that is associated with the user.

15. The method of claim 1 , further comprising:
performing at least one of: an implicit psychological test and an explicit psychological test in the session;
notifying the user when an implicit psychological test is performed to obtain permission from the user to perform the implicit psychological test.

16. The method of claim 15 , further comprising:
notifying the user when an explicit psychological test is performed to obtain permission from the user to perform the explicit psychological test and providing the user with the alternative to perform the implicit psychological test if the user prefers the implicit psychological test over the explicit psychological test.

17. An apparatus for providing emotional care in a session between a user and an electronic conversational agent, comprising:
one or more processors; and
a memory storing computer-executable instructions that, when executed, cause the one or more processors to:
receive a first group of images in the session, the first group of images comprising one or more images associated with the user;
obtain a user profile of the user;
generate a first group of textual descriptions from the first group of images based at least on emotion information in the user profile;
create a first memory record based at least on the first group of images and the first group of textual descriptions;
receive a second group of images in the session;
generate a second group of textual descriptions from the second group of images based at least on the emotion information in the user profile; and
create a second memory record based at least on the second group of images and the second group of textual descriptions.

18. The apparatus of claim 17 , wherein memory storing computer-executable instructions that, when executed, further cause the one or more processors to:
generate a question for inquiring relationship between the first memory record and the second memory record;
provide the question in the session;
receive an answer indicating the relationship in the session; and
create a joint memory record based at least on the first memory record, the second memory record and the relationship.

19. The apparatus of claim 17 , wherein memory storing computer-executable instructions that, when executed, further cause the one or more processors to:
perform at least one of: an implicit psychological test and an explicit psychological test in the session; and
notify the user when an implicit psychological test is performed to obtain permission from the user to perform the implicit psychological test.

20. The apparatus of claim 19 , wherein memory storing computer-executable instructions that, when executed, further cause the one or more processors to:
notify the user when an explicit psychological test is performed to obtain permission from the user to perform the explicit psychological test and provide the user with the alternative to perform the implicit psychological test if the user prefers the implicit psychological test over the explicit psychological test.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wu Xianchao
• Hamura Daigo
• Wang Yongdong

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/454 Primary

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• A61B5/165

  Evaluating the state of mind, e.g. depression, anxiety · CPC title