System and method for monitoring and quality evaluation of perishable food items

Patent metadata

Field	Value
Publication number	US-11488017-B2
Application number	US-202016783755-A
Country	US
Kind code	B2
Filing date	Feb 6, 2020
Priority date	Feb 6, 2019
Publication date	Nov 1, 2022
Grant date	Nov 1, 2022

Abstract

This disclosure relates generally to a system and method for monitoring and quality evaluation of perishable food items in quantitative terms. Current technology provides limited capability for controlling environmental conditions surrounding the food items in real-time or any quantitative measurement for the degree of freshness of the perishable food items. The disclosed systems and methods facilitate in quantitative determination of freshness of food items by utilizing sensor data and visual data obtained by monitoring the food item. In an embodiment, the system utilizes a pre-trained CNN model and a RNN model, where the pertained CNN model is further fine-tined while training the RNN model to provide robust quality monitoring of the food items. In another embodiment, a rate kinetic based model is utilized for determining reaction rate order of the food item at a particular post-harvest stage of the food item so as to determine the remaining shelf life thereof.

First claim

What is claimed is:

1. A processor implemented method, comprising:
obtaining input data comprising visual data and sensory data associated with a food item enclosed in a networked framework, via one or more hardware processors, wherein the visual data and sensory data are time-series data and comprises characteristics indicative of freshness of the food item at a plurality of lifecycle stages;
obtaining, via the one or more hardware processors, a food freshness vector using the input data and one or more machine learning (ML) models, wherein obtaining the food freshness vector comprises:
generating, by a pre-trained convolution neural network (CNN) model, a first vector embedding of the food item at a time-instance using the visual data, the pre-trained CNN model trained as a generic food item classifier using a plurality of images comprising the visual data of a plurality of food items for a plurality of time-instances associated with the plurality of lifecycle stages;
concatenating the first vector embedding and a second vector embedding to obtain a concatenated vector embedding at the time-instance, wherein the second vector embedding obtained from the sensory data of the input data;
obtaining, by fine-tuning the pre-trained CNN model along with the training of a Recurrent Neural Network (RNN), a third vector embedding associated with the food item at the time instance using the concatenated vector embedding, wherein the third vector embedding indicative of a lifecycle stage of the food item at the time instance, wherein the RNN is trained using the time series data of the visual data and the sensory data of the food item aging over a period of time, and comparing, using vector similarity measure, a food freshness vector of the food item at the lifecycle stage from amongst the plurality of lifecycle stages with a digital signature of the food item, via the one or more hardware processors, wherein the food freshness vector of the food item obtained by feeding the visual input of the food item to the fine-tuned CNN model, and wherein the digital signature of the food item is a digitized vector representation of the food item, indicative of freshness of the food item at a target lifecycle stage.

2. The method of claim 1 , wherein during the training of the RNN model, the pre-trained CNN model is fine-tuned to obtain a fine-tuned CNN model, the fine-tuned CNN model being a weight shared model such that a plurality of weights associated with the pre-trained CNN model are updated by a plurality of gradients received from the plurality of time-instances of the RNN model.

3. The method of claim 2 , wherein the digital signature of the food item is obtained by feeding visual input of the target food item to the fine-tuned CNN model.

4. The method of claim 1 , wherein the sensory data comprises weight of the food item, weight loss of the food item, moisture content in the food item, moisture loss during the storage, concentration of specific compound in the food item, and concentration of specific gas such as Carbon dioxide (CO2), ethylene (C2H4), ammonia (NH3) released by the food item.

5. The method of claim 1 , further comprises collecting the visual data and the sensory data using at least one of an invasive and a non-invasive technique, the invasive techniques comprises use of laboratory methods to calculate different food compositional parameters including at least one of sugar, starch, fat, protein, vitamins and antioxidants, and the non-invasive technique comprises at least one of a plurality of non-invasive sensors, the plurality of non-invasive sensors comprises gas sensors, acoustics, optical sensors, and near infrared sensor.

6. A system ( 300 ) comprising:
one or more memories ( 304 ); and
one or more hardware processors ( 302 ), the one or more memories ( 304 ) coupled to the one or more hardware processors ( 302 ), wherein the one or more hardware processors ( 302 ) are configured to execute programmed instructions stored in the one or more memories ( 304 ), to:
obtain input data comprising visual data and sensory data associated with a food item enclosed in a networked framework, wherein the visual data and sensory data are time-series data and comprises characteristics indicative of freshness of the food item at a plurality of lifecycle stages;

obtain a food freshness vector using the input data and one or more machine learning (ML) models, wherein obtaining the food freshness vector comprises:
generate, by a pre-trained convolution neural network (CNN) model, a first vector embedding of the food item at a time-instance using the visual data, the pre-trained CNN model trained as a generic food item classifier using a plurality of images comprising the visual data of a plurality of food items for a plurality of time-instances associated with the plurality of lifecycle stages;
concatenate the first vector embedding and a second vector embedding to obtain a concatenated vector embedding at the time-instance, wherein the second vector embedding obtained from the sensory data of the input data;
obtain, by fine-tuning the pre-trained CNN model along with the training of a Recurrent Neural Network (RNN), a third vector embedding associated with the food item at the time instance using the concatenated vector embedding, wherein the third vector embedding indicative of a lifecycle stage of the food item at the time instance, wherein the RNN is trained using the time series data of the visual data and the sensory data of the food item aging over a period of time, and
compare, using vector similarity measure, the food freshness vector of the food item at the lifecycle stage from amongst the plurality of lifecycle stages with a digital signature of the food item, wherein the food freshness vector of the food item obtained by feeding the visual input of the food item to the fine-tuned CNN model, and wherein the digital signature of the food item is a digitized vector representation of the food item, indicative of freshness of the food item at a target lifecycle stage.

7. The system of claim 6 , wherein the one or more hardware processors are further configured by the instructions to fine-tune the pre-trained CNN model during the training of the RNN model, to obtain a fine-tuned CNN model, the fine-tuned CNN model being a weight shared model such that a plurality of weights associated with the pre-trained CNN model are updated by a plurality of gradients received from the plurality of time-instances of the RNN model.

8. The system of claim 7 , wherein the one or more hardware processors are further configured by the instructions to obtain the digital signature of the food item by feeding visual input of the target food item to the fine-tuned CNN model.

9. The system of claim 6 , wherein the sensory data comprises weight of the food item, weight loss of the food item, moisture content in the food item, moisture loss during the storage, concentration of specific compound in the food item, and concentration of specific gas such as Carbon dioxide (CO2), ethylene (C2H4,), ammonia (NH3) released by the food item.

10. The system of claim 6 , wherein the one or more hardware processors are further configured by the instructions to collect the visual data and the sensory data using at least one of an invasive and a non-invasive technique, the invasive techniques comprises use of laboratory methods to calculate different food compositional parameters including at least one of sugar, starch, fat, protein, vitamins and antioxidants, and the non-invasive technique comprises at least one of a plurality of non-invasive sensors, the plurality of non-invasive sensors comprises gas sensors, acoustics, optical sensors, and nuclear magnetic se

Assignees

• Tata Consultancy Services Ltd

Inventors

• Rai Beena
• Dutta Jayita
• Deshpande Parijat
• Kausley Shankar Balajirao
• Karande Shirish Subhash
• Patwardhan Manasi Samarth
• Deshmukh Shashank Madhukar

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G16C20/70

  Machine learning, data mining or chemometrics · CPC title

• G16C20/30

  Prediction of properties of chemical compounds, compositions or mixtures · CPC title

• G06Q10/0832 Primary

  Special goods or special handling procedures, e.g. handling of hazardous or fragile goods · CPC title