Emotion-aware smart refrigerator

Patent metadata

Field	Value
Publication number	US-12167906-B2
Application number	US-202117143937-A
Country	US
Kind code	B2
Filing date	Jan 7, 2021
Priority date	Jan 7, 2021
Publication date	Dec 17, 2024
Grant date	Dec 17, 2024

Abstract

An emotion-aware smart refrigerator is provided which may include various sensors and presentation mechanisms to identify a user's interest in item(s) contained within the emotion-aware smart refrigerator via an evoked physiological response, such as a visual evoked potential (VEP). The user's level of interest may also be gauged by the emotion-aware smart refrigerator. With consideration given to the user's VEP as well as the user's level of interest, and other considerations such as the user's diet, desired health goals, dietary restrictions, etc., the emotion-aware smart refrigerator may present suggested items and item amounts to the user.

First claim

What is claimed is:

1. A method, comprising:
detecting presence of a user proximate to a refrigerator;
presenting one or more items contained in the refrigerator as visual stimuli displayed on a display of the refrigerator, the visual stimuli prompting a first physiological response from the user regarding the one or more items;
detecting and analyzing the user's first physiological response to the visual stimuli, wherein detecting the user's first physiological response comprises sensing the user's brainwave activity via at least one contactless brainwave sensor configured to wirelessly monitor the user's brainwave activity, and wherein the first physiological response comprises a visual evoked potential (VEP) response;
selecting an item of the one or more items based on analyzing the first physiological response to the visual stimuli;
measuring a second physiological response of the user to the visual stimuli via an ultrasonic transducer or a camera;
determining a level of interest of the user in the selected item based on the second physiological response; and
presenting, via the display of the refrigerator, targeted information regarding the selected one or more items based on the determined level of interest.

2. The method of claim 1 , wherein presenting the visual stimuli comprises presenting one or more images representative of the one or more items to the user on the display, and wherein the presented one or more images oscillate at unique frequencies.

3. The method of claim 1 , wherein presenting the visual stimuli comprises presenting one or more images representative of the one or more items to the user on the display in accordance with a particular order.

4. The method of claim 1 , wherein the one or more items correspond to a current inventory of items contained in the refrigerator.

5. The method of claim 1 , wherein analyzing the user's VEP response comprises correlating the user's brainwave activity to identify the selected item of the one or more items as indicative of an item of interest to the user.

6. The method of claim 1 , wherein measuring the second physiological response of the user comprises determining heart rate activity of the user.

7. The method of claim 6 , wherein determining a level of interest of the user in the selected item based on the measured second physiological response comprises:
comparing the determined heart rate activity to a threshold; and
determining the level of interest of the user based on a difference between the determined heart rate activity and the threshold, wherein a larger difference between the determined heart rate activity and the threshold corresponds to a greater level of interest.

8. The method of claim 1 , wherein measuring the second physiological response of the user comprises determining at least one of body movements and facial expressions.

9. The method of claim 1 , wherein presenting the targeted information comprises presenting a recommended amount of the selected item of one or more items to the user.

10. The method of claim 9 , wherein the recommended amount of the selected item is based on the determined level of interest.

11. The method of claim 1 , wherein presenting the targeted information comprises assessing at least one of an amount of the selected item of the one or more items and an alternative item to the selected item of the one or more items based on at least one of a user profile, one or more learned characteristics of the user, one or more preferences indicated by the user, a day, and a time of day.

12. A refrigerator, comprising:
a presence sensor detecting presence of a user proximate to the refrigerator;
a visual stimulation generation component to present one or more items contained in the refrigerator as visual stimuli on a display of the refrigerator prompting a visual evoked potential (VEP) response from the user regarding one or more items contained in the refrigerator;
a contactless brainwave sensor to detect the user's VEP response to the visual stimuli;
a processor to analyze the user's VEP response to the visual stimuli and select an item of the one or more items based on analyzing the user's VEP response; and
an ultrasonic transducer to measure an interest level response of the user;
wherein the processor further determines a level of interest of the user in the selected item based on the interest level response of the user, and
wherein the visual stimulation generation component further presents targeted information regarding the selected item of the one or more items based on the determined level of interest.

13. The refrigerator of claim 12 , wherein the visual stimulation generation component presents the visual stimuli by presenting one or more images representative of the one or more items to the user on the display, and wherein the presented one or more images oscillate at unique frequencies.

14. The refrigerator of claim 12 , further comprising an inventory tracking component tracking and maintaining a current inventory of items contained in the refrigerator.

15. The refrigerator of claim 12 , further comprising a processor to correlate the user's brainwave activity evidenced by the user's VEP response to identify the selected item of the one or more items as indicative of an item of interest to the user.

16. The refrigerator of claim 12 , wherein to measure the interest level response of the user, the ultrasonic transducer determines a heart rate of the user.

17. The refrigerator of claim 16 , wherein determining a level of interest of the user in the selected item based on the measured interest level response comprises:
comparing the determined heart rate activity to a threshold; and
determining the level of interest of the user based on a difference between the determined heart rate activity and the threshold, wherein a larger difference between the determined heart rate activity and the threshold corresponds to a greater level of interest.

18. The refrigerator of claim 16 , wherein presenting the targeted information comprises presenting a recommended amount of the selected item of one or more items to the user.

19. The refrigerator of claim 18 , wherein the recommended amount of the selected item is based on the determined level of interest.

20. The refrigerator of claim 12 , wherein the visual stimulation generation component presents the targeted information responsive to assessing at least one of an amount of the selected item of the one or more items and an alternative item to the selected item of the one or more items based on at least one of a user profile, one or more learned characteristics of the user, one or more preferences indicated by the user, a day, and a time of day.

Assignees

• Toyota Eng & Mfg North America

Inventors

• Dede Ercan M
• Farooq Muhamed K

Classifications

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• F25D2700/06

  Sensors detecting the presence of a product · CPC title

• G06F3/015

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title

• F25D2700/04

  Sensors detecting the presence of a person · CPC title

• A61B5/165

  Evaluating the state of mind, e.g. depression, anxiety · CPC title