Method and system for foodstuff identification

Patent metadata

Field	Value
Publication number	US-2022015572-A1
Application number	US-202117376535-A
Country	US
Kind code	A1
Filing date	Jul 15, 2021
Priority date	Jul 15, 2020
Publication date	Jan 20, 2022
Grant date	—

Abstract

A method for foodstuff identification can include: detecting a trigger event; sampling a measurement set; optionally determining candidate measurements for subsequent analysis based on the measurement set; optionally determining a set of food parameter values from the measurements; optionally selecting a food parameter value for use; determining a cooking instruction based on the food parameter value; automatically operating the appliance based on the cooking instructions; optionally determining a foodstuff trajectory relative to the cook cavity; optionally training one or more modules; and/or any other suitable elements.

First claim

1 . A method, comprising:
detecting a start trigger event at an appliance; sampling a series of measurements in response to start trigger event detection; selecting a candidate measurement set from the series of measurements; calculating a food parameter value using the candidate measurement set; determining a foodstuff trajectory relative to a cook cavity of the appliance using the series of measurements; determining a cooking instruction based on the foodstuff trajectory and the food parameter value; and automatically operating the appliance based on the cooking instruction.

2 . The method of claim 1 , wherein the appliance comprises a door, and wherein the start trigger event is a door open event.

3 . The method of claim 1 , wherein the series of measurements comprise a series of images.

4 . The method of claim 1 , wherein the candidate measurement set comprises a candidate image set, and wherein the candidate image set is selected using motion detection.

5 . The method of claim 4 , wherein a best candidate image of the candidate image set is selected using a search technique, wherein the best candidate image depicts the foodstuff within a region of interest relative to an insertion path into the cook cavity.

6 . The method of claim 1 , wherein the food parameter value is calculated using a trained neural network that is trained to identify foodstuff depicted in a measurement.

7 . The method of claim 1 , wherein the food parameter value is calculated within 200 milliseconds after the start trigger event is detected.

8 . The method of claim 1 , wherein the series of measurements are sampled by an camera system that is mounted above a cook cavity external to the appliance and angled outward.

9 . The method of claim 8 , wherein the appliance comprises an oven, wherein the camera system is mounted to an interior portion of an oven doorframe.

10 . The method of claim 1 , wherein when multiple foodstuff items are within the cook cavity and when the foodstuff trajectory is a removal trajectory, the cooking instruction is determined based on a remainder of the multiple foodstuff items that remain within the cook cavity.

11 . The method of claim 10 , wherein the remainder of the multiple foodstuff items is determined using the food parameter value and prior food parameter values for the foodstuff items of the multiple foodstuff items.

12 . A system, comprising:
an appliance comprising a cook cavity and a camera system mounted above the cook cavity; processing system, configured to: detect a start trigger event at the appliance; sample a series of measurements; select a candidate measurement set from the series of measurements; calculate a food parameter value using the candidate measurement set; determine a foodstuff trajectory relative to the cook cavity using the series of measurements; determine a cooking instruction based on the foodstuff trajectory and food parameter value; and automatically operate the appliance based on the cooking instruction.

13 . The system of claim 12 , wherein the series of measurements comprises a series of images sampled by the camera system.

14 . The system of claim 13 , wherein the camera system comprises a stereo camera.

15 . The system of claim 13 , wherein the camera system is mounted to a top surface of an appliance, and is angled outward relative to the cook cavity.

16 . The system of claim 12 , wherein the food parameter value is a foodstuff identification determined using a trained neural network.

17 . The system of claim 12 , wherein selecting the candidate measurement set comprises selecting a measurement from the series of measurements wherein the foodstuff encompasses at least a predetermined proportion of the measurement.

18 . The system of claim 12 , wherein selecting the candidate measurement set comprises using motion detection to identify a measurement of the series that depicts a foodstuff within a region of interest, and when the measurement is identified, using the measurement to calculate the food parameter value.

19 . The system of claim 12 , wherein the food parameter value is determined based on M of N agreements of food parameter values.

Assignees

• June Life Inc

Inventors

• Cottrell Robert
• Paruchuri Jithendra
• Bhogal Nikhil
• Kumar Nishit
• Wang Wiley

Classifications

• G06F18/2431

  Multiple classes · CPC title

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/68

  Food, e.g. fruit or vegetables · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title