Dynamic feature selection for model generation

Patent metadata

Field	Value
Publication number	US-11599753-B2
Application number	US-201715844991-A
Country	US
Kind code	B2
Filing date	Dec 18, 2017
Priority date	Dec 18, 2017
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

Embodiments generate a model of demand of a product that includes an optimized feature set. Embodiments receive sales history for the product and receive a set of relevant features for the product and designate a subset of the relevant features as mandatory features. From the sales history, embodiments form a training dataset and a validation dataset and randomly select from the set of relevant features one or more optional features. Embodiments include the selected optional features with the mandatory features to create a feature test set. Embodiments train an algorithm using the training dataset and the feature test set to generate a trained algorithm and calculate an early stopping metric using the trained algorithm and the validation dataset. When the early stopping metric is below a predefined threshold, the feature test set is the optimized feature set.

First claim

What is claimed is:

1. A method of generating a model of demand of a product that comprises an optimized feature set, the method comprising:
receiving sales history for the product;
receiving a set of relevant features for the product and designating a subset of the relevant features as mandatory features;
from the sales history, forming a training dataset and a validation dataset;
randomly selecting, from the set of relevant features, one or more optional features to create a set of optional features and including the set of optional features with the mandatory features to create a first feature test set;
training an algorithm using the training dataset and the first feature test set to generate a trained algorithm;
calculating an early stopping metric using the trained algorithm and the validation dataset;
repeating the randomly selecting to create a revised feature test set, the revised feature test set comprising the mandatory feature set and a different set of optional features than the first feature test set;
repeating the training and calculating using the revised feature test set instead of the first feature test set; and
repeating the repeating the randomly selecting and the repeating the training and calculating until the early stopping metric is below a predefined threshold;
wherein when the early stopping metric is below the predefined threshold, the revised feature test set is the optimized feature set.

2. The method of claim 1 , further comprising repeating the method to generate a plurality of optimized feature sets, where each of the optimized feature sets are input into a forecasting algorithm to generate a trained model.

3. The method of claim 1 , wherein the forming the training dataset and the validation dataset comprises randomly selecting a first subset of the sales history as the training dataset, and using a remainder of the sales history as the validation dataset.

4. The method of claim 1 , wherein the algorithm comprises a machine learning algorithm that comprises one of linear regression, Support Vector Machine, or Artificial Neural Networks.

5. The method of claim 1 , wherein the early stopping metric comprises a mean absolute percentage error.

6. The method of claim 2 , further comprising:
training multiple models corresponding to the optimized feature sets using a training set, and using a corresponding validation set to validate each trained model and calculate an error;
calculating model weights for each model;
outputting a model combination comprising for each model a forecast and a weight; and
generating a forecast of future sales based on the model combination.

7. The method of claim 6 , wherein the error is a root-mean-square error (RMSE) and for each model of each training set i, the calculating model weights w(i) comprises:

w
⁡

(
i
)

=

1

1
+

RMSE
⁡

(
i
)

.

8. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a-one or more processors, cause the processors to generate a model of demand of a product that comprises an optimized feature set, the generating comprising:
receiving sales history for the product;
receiving a set of relevant features for the product and designating a subset of the relevant features as mandatory features;
from the sales history, forming a training dataset and a validation dataset;
randomly selecting, from the set of relevant features, one or more optional features to create a set of optional features and including the set of optional features with the mandatory features to create a first feature test set;
training an algorithm using the training dataset and the first feature test set to generate a trained algorithm;
calculating an early stopping metric using the trained algorithm and the validation dataset;
repeating the randomly selecting to create a revised feature test set, the revised feature test set comprising the mandatory feature set and a different set of optional features than the first feature test set;
repeating the training and calculating using the revised feature test set instead of the first feature test set; and
repeating the repeating the randomly selecting and the repeating the training and calculating until the early stopping metric is below a predefined threshold;
wherein when the early stopping metric is below the predefined threshold, the revised feature test set is the optimized feature set.

9. The computer-readable medium of claim 8 , the generating further comprising repeating the generating to generate a plurality of optimized feature sets, where each of the optimized feature sets are input into a forecasting algorithm to generate a trained model.

10. The computer-readable medium of claim 8 , wherein the forming the training dataset and the validation dataset comprises randomly selecting a first subset of the sales history as the training dataset, and using a remainder of the sales history as the validation dataset.

11. The computer-readable medium of claim 8 , wherein the algorithm comprises a machine learning algorithm that comprises one of linear regression, Support Vector Machine, or Artificial Neural Networks.

12. The computer-readable medium of claim 8 , wherein the early stopping metric comprises a mean absolute percentage error.

13. The computer-readable medium of claim 9 , further comprising:
training multiple models corresponding to the optimized feature sets using a training set, and using a corresponding validation set to validate each trained model and calculate an error;
calculating model weights for each model;
outputting a model combination comprising for each model a forecast and a weight; and
generating a forecast of future sales based on the model combination.

14. The computer-readable medium of claim 13 , wherein the error is a root-mean-square error (RMSE) and for each model of each training set i, the calculating model weights w(i) comprises:

w
⁡

(
i
)

=

1

1
+

RMSE
⁡

(
i
)

.

15. A retail sales forecasting system that forecasts demand for a product using an optimized feature set, the system comprising:
one or morea processors coupled to a storage device that implements a demand forecasting module comprising:
receiving sales history for the product;
receiving a set of relevant features for the product and

Assignees

• Oracle Int Corp

Inventors

• Lei Ming
• Popescu Catalin

Classifications

• G06Q30/0202

  Market predictions or forecasting for commercial activities · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F17/10

  Complex mathematical operations {(function generation by table look-up G06F1/03; evaluation of elementary functions by calculation G06F7/544)} · CPC title

• G06V10/98

  Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns · CPC title