Electronic device and controlling method thereof

Patent metadata

Field	Value
Publication number	US-11147488-B2
Application number	US-201916683362-A
Country	US
Kind code	B2
Filing date	Nov 14, 2019
Priority date	Feb 19, 2019
Publication date	Oct 19, 2021
Grant date	Oct 19, 2021

Abstract

An electronic device attached to a user's skin is provided to determine a user emotion based on measured bio-signals. An image corresponding to the determined emotion is displayed. The electronic device includes a skin-attachment apparatus having a bonding unit for bonding to a user's skin and a bio-signal sensor that measures the bio-signals of the user. A display unit and a controller are connected with the skin-attachment apparatus. The controller determines the user emotion based on the bio-signals acquired from the bio-signal sensor and operates the display unit to display at least one of a color and a pattern corresponding to the determined emotion.

First claim

What is claimed is:

1. An electronic device, comprising:
a skin-attachment apparatus configured to include a bonding unit for bonding to a user's skin, a bio-signal sensor configured to measure bio-signals of the user, and a display unit;
a memory configured to store at least one information of color information and pattern information corresponding to each of predetermined emotions; and
a processor configured to be connected with the skin-attachment apparatus, determine a user emotion based on the bio-signals acquired from the bio-signal sensor, and operate the display unit to display at least one of a color and a pattern corresponding to the determined emotion based on the at least one information,
wherein the display unit includes:
a heating circuit; and
a thermochromic pigment layer configured to include at least one thermochromic pigment provided on the heating circuit.

2. The electronic device of claim 1 , wherein the processor is configured to determine the color corresponding to the determined emotion, and operate the heating circuit to generate heat at a temperature corresponding to the determined color.

3. The electronic device of claim 1 , wherein the processor is configured to determine the pattern corresponding to the determined emotion, and operate the heating circuit to generate heat at a region of the heating circuit corresponding to the determined pattern.

4. The electronic device of claim 1 , wherein the processor is configured to determine a facial expression corresponding to the determined emotion, determine a pattern for each body part based on a feature of the body part in the determined facial expression, and operate the display unit to display the pattern among the determined patterns corresponding to the body part to which the skin-attachment apparatus is attached.

5. The electronic device of claim 4 , wherein the skin-attachment apparatus, respectively provided in plural, are attached to the different body parts.

6. The electronic device of claim 1 , wherein the electronic device further includes:
a camera configured to acquire image data of user surroundings.

7. The electronic device of claim 6 , wherein the processor is configured to determine at least one of a facial expression and an action of an object based on image data of the object included in the image data and determine an emotion of the object based on at least one of the determined facial expression and the action.

8. The electronic device of claim 7 , wherein the processor is configured to determine a feedback emotion corresponding to the emotion of the object, and operate the display unit to display at least one of the color and the pattern corresponding to the determined feedback emotion.

9. The electronic device of claim 8 , wherein the processor is configured to perform an arithmetic operation on the emotion of the object using a neural network, determine the feedback emotion corresponding to the emotion of the object based on the information on the operation performed using the neural network, determine a change in the emotion of the object with respect to at least one of the color and the pattern corresponding to the determined feedback emotion based on the image data of the object, and update the neural network based on the determined feedback emotion and the emotion change of the object.

10. A controlling method of an electronic device including a skin-attachment apparatus bonded to a user's skin, a bio-signal sensor configured to measure bio-signals of the user, and a display unit, the method comprising:
determining, by a processor, a user emotion based on the bio-signals acquired from the bio-signal sensor; and
operating, by the processor, the display unit to display at least one of a color and a pattern corresponding to the determined emotion based on stored at least one information of color information and pattern information corresponding to each of predetermined emotions,
wherein the display unit includes:
a heating circuit; and
a thermochromic pigment layer configured to include at least one thermochromic pigment provided on the heating circuit.

11. The method of claim 10 , wherein the operating of the display unit includes:
determining, by the processor, the color corresponding to the determined emotion; and
operating, by the processor, the heating circuit to generate heat at a temperature corresponding to the determined color.

12. The method of claim 10 , wherein the operating of the display unit includes:
determining, by the processor, the pattern corresponding to the determined emotion; and
operating, by the processor, the heating circuit to generate heat at a region of the heating circuit corresponding to the determined pattern.

13. The method of claim 10 , wherein the operating of the display unit includes:
determining, by the processor, a facial expression corresponding to the determined emotion;
determining, by the processor, the pattern for each body part based on a feature of the body part in the determined facial expression; and
operating, by the processor, the display unit to display the pattern among the determined patterns corresponding to the body part to which the skin-attachment apparatus is attached.

14. The method of claim 13 , wherein the skin-attachment apparatus, respectively provided in plural, are attached to different body parts.

15. The method of claim 10 , wherein the electronic device further includes:
a camera configured to acquire image data of user surroundings.

16. The method of claim 15 , further comprising:
determining, by the processor, at least one of a facial expression and an action of an object based on image data of the object included in the image data; and
determining, by the processor, an emotion of the object based on at least one of the determined facial expression and the action.

17. The method of claim 16 , further comprising:
determining, by the processor, a feedback emotion corresponding to the emotion of the object;
operating, by the processor, the display unit to display at least one of the color and the pattern corresponding to the determined feedback emotion.

18. The method of claim 17 , wherein the determining of the feedback emotion includes:
performing, by the processor, an arithmetic operation on the emotion of the object using a neural network;
determining, by the processor, the feedback emotion corresponding to the emotion of the object based on the information on the operation performed using the neural network;
determining, by the processor, a change in the emotion of the object with respect to at least one of the color and the pattern corresponding to the determined feedback emotion based on the image data of the object; and
updating, by the processor, the neural network based on the determined feedback emotion and the emotion change of the object.

Assignees

• Hyundai Motor Co Ltd
• Kia Motors Corp

Inventors

• Lee Jia
• Youn Seok-Young
• Han Jimin
• Kim Kye Yoon
• Woo Seunghyun

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• A61B5/165 Primary

  Evaluating the state of mind, e.g. depression, anxiety · CPC title

• A61B5/742

  using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339) · CPC title

• G06N3/08

  Learning methods · CPC title