Food monitoring apparatus, refrigerator including the same, and operating method thereof

Patent metadata

Field	Value
Publication number	US-11758091-B2
Application number	US-202117366695-A
Country	US
Kind code	B2
Filing date	Jul 2, 2021
Priority date	Dec 11, 2020
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

Provided is a food monitoring apparatus including at least one light source configured to selectively radiate light of a first wavelength band and light of a second wavelength band that is different from the first wavelength band to food, at least one image sensor configured to obtain a visible image of the food and a hyperspectral image of the food based on light scattered, emitted, or reflected from the food, and at least one processor configured to obtain first information of the food based on the visible image, and to obtain second information of the food based on the first information and the hyperspectral image.

First claim

What is claimed is:

1. A food monitoring apparatus comprising:
at least one light source configured to selectively radiate light of a first wavelength band and light of a second wavelength band that is different from the first wavelength band to food;
at least one image sensor configured to obtain a visible image of the food and a hyperspectral image of the food based on light scattered, emitted, or reflected from the food; and
at least one processor configured to obtain first information of the food based on the visible image, and to obtain second information of the food based on the first information and the hyperspectral image,
wherein the at least one processor is further configured to obtain at least one parameter representing a content of an indicator material included in the food from the hyperspectral image, the indicator material emitting fluorescence based on the light radiated by the at least one light source, and
wherein the at least one processor is further configured to correct, based on the first information of the food obtained based on the visible image, a value of the at least one parameter obtained from the hyperspectral image, and obtain the second information based on the corrected value of the at least one parameter.

2. The food monitoring apparatus of claim 1 , wherein the at least one processor is further configured to further correct, based on the first information, at least one of the hyperspectral image, and a correlation between the at least one parameter and the second information.

3. The food monitoring apparatus of claim 2 , wherein the first information includes at least one of a type of the food and a part of the food, and
wherein the at least one processor is further configured to selectively apply a weight to the at least one parameter based on the first information, and obtain the second information by analyzing the hyperspectral image based on the at least one parameter to which the weight is applied.

4. The food monitoring apparatus of claim 2 , wherein the first information includes a packaging condition of the food, and
wherein the at least one processor is further configured to correct the at least one parameter based on the packaging condition, and obtain the second information based on the corrected at least one parameter.

5. The food monitoring apparatus of claim 2 , wherein the at least one parameter includes at least one of 1,4-dihydronicotinamide adenine dinucleotide (NADH) content, porphyrin content, flavin adenine dinucleotide (FAD) content, and collagen content included in the food.

6. The food monitoring apparatus of claim 1 , wherein the first information includes a location of the food in the food monitoring apparatus, and
wherein the at least one processor is further configured to:
control the at least one light source to radiate the light of the second wavelength band to an area of the food monitoring apparatus corresponding to the location of the food; and
obtain the hyperspectral image from light received from the area of the food monitoring apparatus corresponding to the location of the food.

7. The food monitoring apparatus of claim 1 , wherein the first wavelength band ranges from 400 nm to 700 nm,
wherein the second wavelength band ranges from 250 nm to 400 nm, and
wherein the at least one image sensor is further configured to measure light in a wavelength band ranging from 300 nm to 700 nm.

8. A refrigerator comprising:
a main body comprising a storage space;
a door configured to open and close the storage space; and
the food monitoring apparatus according to claim 1 that is disposed in the storage space.

9. The refrigerator of claim 8 , wherein the second information includes at least one of freshness, maturity, and spoilage of the food, and
wherein the at least one processor is further configured to determine at least one of whether the food is edible, an intakeable period of the food, an optimum intake time of the food, and a cooking method of the food based on the at least one of the freshness, maturity, and spoilage of the food.

10. The refrigerator of claim 8 , further comprising a controller comprising at least one processor and configured to control at least one of temperature, humidity, and gas composition ratio in the storage space.

11. The refrigerator of claim 10 , wherein the second information includes maturity of the food, and
wherein the at least one processor is further configured to:
determine whether acceleration or deceleration of maturation of the food is necessary to reach a target maturation at a preset intake time of the food based on the maturity of the food; and
control the at least one of temperature, humidity, and gas composition ratio in the storage space by the controller.

12. The refrigerator of claim 11 , wherein, when it is determined that acceleration or deceleration of maturation of the food is necessary, the at least one processor is further configured to increase or decrease a temperature in the storage space from a preset temperature during at least a portion of a period between a time when the maturity of the food is determined and the preset intake time.

13. The refrigerator of claim 8 , further comprising a display device disposed on the door, the display device configured to output the second information received from the food monitoring apparatus.

14. The refrigerator of claim 8 , further comprising a communication interface configured to communicate with an external device to transmit the second information to the external device.

15. A method of operating a food monitoring apparatus, the method comprising:
selectively radiating light of a first wavelength band and light of a second wavelength band that is different from the first wavelength band to food by at least one light source;
obtaining a visible image of the food and a hyperspectral image of the food based on light scattered, emitted, or reflected from the food by at least one image sensor;
obtaining first information of the food based on the visible image; and
obtaining second information of the food based on the first information and the hyperspectral image,
wherein the obtaining the second information includes:
obtaining at least one parameter representing a content of an indicator material included in the food from the hyperspectral image, the indicator material emitting fluorescence based on the light radiated by the at least one light source;
correcting, based on the first information of the food obtained based on the visible image, a value of the at least one parameter obtained from the hyperspectral image; and
obtaining the second information based on the corrected value of the at least one parameter.

16. The method of claim 15 , wherein the obtaining of the second information includes correcting, based on the first information, at least one of the hyperspectral image, and a correlation between the at least one parameter and the second information.

17. The method of claim 16 , wherein the first information includes at least one of a type of the food and a part of the food, and
wherein the obtaining of the second information includes selectively applying a weight to the at least one parameter based on the first information, and obtaining the second information by analyzing the hyperspectral image based on the at least one parameter to which the weight is applied.

18. The method of claim 16 , wherein the first information includes a packaging condition of the food, and
wherein the obtaining of the second information includes correcting the at least

Assignees

• Samsung Electronics Co Ltd
• Univ Chung Ang Ind Acad Coop Found

Inventors

• Lee Suyeon
• Han Jeongsu
• Kim Unjeong
• Son Hyungbin
• Choi Junhoe

Classifications

• G01N21/84 Primary

  Systems specially adapted for particular applications · CPC title

• G01N21/31 Primary

  Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry {(G01N21/72 takes precedence)} · CPC title

• G01J3/2823 Primary

  Imaging spectrometer · CPC title

• H04N7/18 Primary

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title

• G01N33/02

  Food · CPC title