Artificial intelligence device and operating method thereof

Patent metadata

Field	Value
Publication number	US-2024257542-A1
Application number	US-202318104422-A
Country	US
Kind code	A1
Filing date	Feb 1, 2023
Priority date	Feb 1, 2023
Publication date	Aug 1, 2024
Grant date	—

Abstract

An artificial intelligence range hood including a main body; a camera disposed at a lower end of the main body and configured to capture an image of a cooktop located under the main body; a display located on a front surface of the main body; a memory storing an object recognition model; and a processor configured to control the camera to capture the image of the cooktop, generate object recognition information for recognizing cooking objects included the captured image of the cooktop using the object recognition model stored in the memory, set a user region of interest of the captured image corresponding to a recognized cooking object designated by the generated object information, control an operation of the camera to photograph the user region of interest of the captured image, and control the display to display the photographed user region of interest of the captured image.

First claim

What is claimed is:

1 . An artificial intelligence range hood comprising:
a main body; a camera disposed at a lower end of the main body and configured to capture an image of a cooktop located under the main body; a display located on a front surface of the main body; a memory storing an object recognition model; and a processor configured to: control the camera to capture the image of the cooktop, generate object recognition information for recognizing cooking objects included the captured image of the cooktop using the object recognition model stored in the memory, set a user region of interest of the captured image corresponding to a recognized cooking object designated by the generated object information, control an operation of the camera to photograph the user region of interest of the captured image, and control the display to display the photographed user region of interest of the captured image.

2 . The artificial intelligence range hood of claim 1 , wherein the object recognition information comprises object identification information and object location information on the cooking objects included in the captured image, and
wherein the object identification information designates at least one of a cooking vessel, a food, a cooking utensil, and a user's hand.

3 . The artificial intelligence range hood of claim 1 , wherein the recognized cooking object comprises least one of one or more cooking zones, cooking utensils, and food being cooked, and
wherein the user region of interest includes the recognized cooking object.

4 . The artificial intelligence range hood of claim 2 , wherein the processor controls the operation of the camera to photograph the user region of interest of the captured image in response to a user input including a time-lapse photographing speed and a photographing mode.

5 . The artificial intelligence range hood of claim 4 , wherein the processor calculates a ratio of a region occupied by a food and a ratio of a region occupied by an object other than the food, respectively, within the user region of interest using the object identification information and the object location information, increases the time-lapse photographing speed when the ratio of the region occupied by the food is greater than a preset first threshold, and decreases the time-lapse photographing speed when the ratio of the region occupied by the food is less than the first threshold and the ratio of the region occupied by the object other than the food is greater than a preset second threshold, and
wherein the first threshold is set as a reference value for determining the region occupied by the food within the user region of interest, and the second threshold is set as a reference value for determining the region occupied by the object other than the food within the user region of interest.

6 . The artificial intelligence range hood of claim 2 , wherein the memory further stores a food identification model, and
wherein when a food is included in the object identification information, the processor generates food identification information using the food identification model, and the food identification information identifies a type of the food included in the captured image.

7 . The artificial intelligence range hood of claim 1 , wherein the processor controls the display to display color correction filter lists, and applies a user-selected color correction filter to the user region of interest of the captured image.

8 . The artificial intelligence range hood of claim 4 , wherein the memory further stores a motion identification model,
wherein when a user's hand is included in the object identification information, the processor generates motion identification information using the motion identification model, wherein the motion identification information identifies a motion type of the user's hand included in the captured image, and wherein the motion identification information comprising at least one of motion recognition start, thumb-up, thumb-down, zoom-in and zoom-out motions.

9 . The artificial intelligence range hood of claim 8 , wherein the processor changes a motion recognition mode to an ON state when the motion identification information is a motion recognition start motion, increases or decreases the time-lapse photographing speed when the motion identification information is one of a thumb-up and a thumb-down motion and the motion recognition mode is in the ON state, and zooms in or out a screen on which the user region of interest is displayed when the motion identification information is one of a zoom-in motion and a zoom-out motion and the motion recognition mode is in the ON state.

10 . The artificial intelligence range hood of claim 1 , further comprising:
a plurality of directional microphones located at the lower end of the main body, wherein the processor controls the directional microphones to directionally record sounds from the cooking object included in the users region of the interest.

11 . The artificial intelligence range hood of claim 1 , wherein the processor calculates a similarity between a current frame and a previous frame from the captured image, determines the current frame as a first change point frame when the calculated similarity is less than a preset threshold similarity, and generates a highlight clip including the first change point frame.

12 . The artificial intelligence range hood of claim 11 , wherein the processor calculates the similarity between the current frame and the previous frame based on at least one of a color, an edge, a histogram, a correlation, and a motion vector of an optical flow extracted from image data for the current frame and the previous frame.

13 . The artificial intelligence range hood of claim 11 , wherein the processor generates first and second object recognition information for the current frame and the previous frame to calculate ratios occupied by a food region within the user region of interest, respectively, and increases or decreases the threshold similarity based on the calculated ratios, respectively.

14 . The artificial intelligence range hood of claim 2 , wherein the processor generates a recipe database based on recipe information collected through an open application program interface and stores the generated recipe database in the memory,
wherein the memory further stores a plurality of recipes including one or more items of a dish name, a cooking method, ingredient information, images illustrating the cooking method, and text describing the cooking method in the recipe database, and wherein the processor further generates a cooking vessel item corresponding to the cooking method.

15 . The artificial intelligence range hood of claim 14 , wherein the memory further stores a cooking vessel identification model,
wherein the processor generates cooking vessel identification information using the cooking vessel identification model when a cooking vessel is included in the object identification information, and wherein the cooking vessel identification information identifies a type of the cooking vessel included in the captured image.

16 . The artificial intelligence range hood of claim 15 , wherein the processor retrieves the cooking vessel item corresponding to the cooking vessel identification information from the recipe database to extract one or more dish names related to the retrieved cooking vessel item, and controls the display to display a list of recommended dishes including the extracted one or more dish names.

17 . The artificia

Assignees

• Lg Electronics Inc

Inventors

• Jeong Seoyoung
• Jeon Seongik
• Baek Haesol
• Lee Jeehoon

Classifications

• G06V2201/07

  Target detection · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• F24C15/20

  Removing cooking fumes (parts, details or accessories of cooking-vessels for withdrawing or condensing cooking vapours from such vessels A47J36/38) · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06V10/82

  using neural networks · CPC title