Systems and methods for cookware detection

Patent metadata

Field	Value
Publication number	US-9412048-B2
Application number	US-201414257282-A
Country	US
Kind code	B2
Filing date	Apr 21, 2014
Priority date	Apr 21, 2014
Publication date	Aug 9, 2016
Grant date	Aug 9, 2016

Abstract

Systems and methods for cookware detection are provided. One example system includes a vision sensor positioned so as to collect imagery depicting a cooktop. The system includes one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include obtaining a frame of imagery depicting the cooktop and using a classifier to classify the frame of imagery as either depicting cookware or not depicting cookware. When the frame is classified as depicting cookware, the operations include performing one or more operations in response to the detected cookware.

First claim

What is claimed is:

1. A system for detecting cookware, the system comprising:
a vision sensor positioned so as to collect imagery depicting a cooktop;
one or more processors; and
one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining a frame of imagery depicting the cooktop;
performing background subtraction for the frame of imagery with respect to a reference frame of imagery depicting the cooktop without any objects placed thereon;
after performing the background subtraction for the frame of imagery, using a classifier to classify the frame of imagery as either depicting cookware or not depicting cookware; and
when the frame is classified as depicting cookware, performing one or more operations in response to the detected cookware.

2. The system of claim 1 , wherein the operations further comprise, prior to using the classifier to classify the frame of imagery as either depicting cookware or not depicting cookware, calibrating the system to identify one or more locations respectively associated with one or more burners included in the cooktop.

3. The system of claim 2 , wherein calibrating the system to identify one or more locations respectively associated with one or more burners included in the cooktop comprises:
obtaining a reference frame of imagery depicting the cooktop without any objects placed thereon;
obtaining one or more calibration frames of imagery depicting the cooktop with one or more objects respectively positioned at the one or more locations respectively associated with the one or more burners;
performing background subtraction for the one or more calibration frames of imagery with respect to the reference frame of imagery to identify the one or more locations.

4. The system of claim 3 , wherein:
the one or more calibration frames of imagery comprise a single calibration frame of imagery;
the one or more objects respectively positioned at the one or more locations respectively associated with the one or more burners comprise a plurality of objects respectively positioned at a plurality of locations respectively associated with a plurality of burners included in the cooktop; and
performing background subtraction for the single calibration frame of imagery with respect to the reference frame of imagery further comprises performing object segmentation to identify the plurality of locations.

5. The system of claim 2 , wherein calibrating the system to identify one or more locations respectively associated with one or more burners included in the cooktop comprises:
obtaining a reference frame of imagery depicting the cooktop without any objects placed thereon;
obtaining one or more calibration frames of imagery depicting the cooktop with the one or more burners energized;
performing background subtraction for the one or more calibration frames of imagery with respect to the reference frame of imagery to identify the one or more locations.

6. The system of claim 1 , wherein:
the operations further comprise segmenting the frame of imagery according to a plurality of burner locations respectively associated with a plurality of burners included in the cooktop, such that a plurality of frame segments of formed that respectively correspond to the plurality of burners; and
using the classifier to classify the frame of imagery as either depicting cookware or not depicting cookware comprises using the classifier to individually classify each of the plurality of frame segments as either depicting cookware or not depicting cookware, such that cookware detection is performed on a burner-by-burner basis.

7. The system of claim 1 , wherein the classifier comprises a support vector machine.

8. The system of claim 1 , wherein one or more operations in response to the detected cookware comprise performing a boil detection algorithm.

9. The system of claim 1 , wherein one or more operations in response to the detected cookware comprise transmitting a communication to the cooktop that informs the cooktop of the presence of the detected cookware.

10. The system of claim 9 , wherein the communication further identifies one or more of a plurality of burners included in the cooktop upon which the detected cookware is positioned.

11. The system of claim 1 , wherein:
the vision sensor comprises a wide angle VGA resolution camera; and
the one or more processors and the one or more non-transitory computer-readable media comprise a single System-On-Chip.

12. A method for detecting cookware, the method comprising:
collecting, by a vision sensor, one or more frames comprising imagery depicting a cooktop;
segmenting each of the one or more frames into a plurality of frame segments based on a plurality of locations respectively associated with a plurality of burners included in the cooktop;
classifying, by a binary classifier, each of the plurality of frame segments as either depicting an item of cookware or not depicting an item of cookware, such that cookware detection is performed on a burner-by-burner basis; and
when at least one of the frame segments is classified as depicting an item of cookware, performing one or more operations responsive to the detected item of cookware.

13. The method of claim 12 , further comprising, prior to classifying each of the one or more frames, calibrating the system to identify one or more locations respectively associated with one or more burners included in the cooktop, wherein the calibration is performed by comparing reference imagery depicting the cooktop without objects placed thereon with calibration imagery depicting the cooktop with one or more objects respectively placed on the one or more burners.

14. The method of claim 12 , further comprising, prior to said segmenting and said classifying, performing a calibration routine to identify the plurality of locations respectively associated the plurality of burners included in the cooktop.

15. The method of claim 14 , wherein performing the calibration routine comprises:
obtaining a reference frame of imagery depicting the cooktop without any objects placed thereon;
obtaining one or more calibration frames of imagery depicting the cooktop with a plurality of objects respectively positioned at the plurality of locations respectively associated with the plurality of burners;
performing background subtraction for the one or more calibration frames of imagery with respect to the reference frame of imagery to identify the plurality of locations.

16. The method of claim 15 , wherein:
the one or more calibration frames of imagery comprise a single calibration frame of imagery; and
performing background subtraction for the single calibration frame of imagery with respect to the reference frame of imagery further comprises performing object segmentation to identify the plurality of locations.

17. The method of claim 14 , wherein performing the calibration routine comprises:
obtaining a reference frame of imagery depicting the cooktop without any objects placed thereon;
obtaining one or more calibration frames of imagery depicting the cooktop with the plurality of burners energized; and
performing background subtraction for the one or more calibration frames of imagery with respect to the reference frame of imagery to identify the plurality of locations.

18. A system for detecting cookware, the system comprising:
a vision sensor positioned so as to collect imagery depicting a cooktop;

Assignees

• Gen Electric
• Haier Us Appliance Solutions Inc

Inventors

• Guan Li
• Smith Charles Ray

Classifications

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06V20/00 Primary

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title

• G06K9/00624

  Physics · mapped topic

• F24C7/083

  on tops, hot plates · CPC title

• G06K9/627 Primary

  Physics · mapped topic