Recurrent neural network-based artificial intelligence system for time series predictions

Patent metadata

Field	Value
Publication number	US-10936947-B1
Application number	US-201715417070-A
Country	US
Kind code	B1
Filing date	Jan 26, 2017
Priority date	Jan 26, 2017
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

At a network-accessible artificial intelligence service for time series predictions, a recurrent neural network model is trained using a plurality of time series of demand observations to generate demand forecasts for various items. A probabilistic demand forecast is generated for a target item using multiple executions of the trained model. Within the training set used for the model, the count of demand observations of the target item may differ from the count of demand observations of other items. A representation of the probabilistic demand forecast may be provided via a programmatic interface.

First claim

What is claimed is:

1. A system, comprising:
one or more computing devices of a recurrent neural-network based artificial intelligence service for time series predictions;
wherein the one or more computing devices are configured to:
obtain an indication of a first data set comprising a respective time series of demand observations for a plurality of items;
train a recurrent neural network model using a plurality of time series of the first data set, wherein the recurrent neural network is trained to make probabilistic demand forecasts for a plurality of different items, and wherein training of the recurrent neural network model comprises performing a compensation operation to the plurality of times series of the first data set to compensate for differences between respective distributions of the demand observations for different items within the first data set;
receive indications of different target items for which respective probabilistic demand forecasts are to be generated, wherein the first data set does not include a time series comprising demand observations of at least one of the target items;
obtain respective probabilistic demand forecasts for the different target items using a plurality of executions of the same recurrent neural network model, wherein the respective probabilistic demand forecasts comprise respective aggregations of individual demand forecasts indicated by respective executions of the plurality of executions; and
provide, via a programmatic interface, respective representations of the probabilistic demand forecasts.

2. The system as recited in claim 1 , wherein to obtain the probabilistic demand forecasts, the one or more computing devices are configured to:
predict, using the recurrent neural network model, one or more parameters of a first demand distribution corresponding to a first time step; and
utilize a sample from the first demand distribution as input of the model, wherein a second set of nodes of the model is used to predict parameters of a second demand distribution corresponding to a second time step.

3. The system as recited in claim 1 , wherein the compensation operation comprises:
generating one or more sub-sequences of a time series of the first data set; and
utilizing the one or more sub-sequences as respective training examples.

4. The system as recited in claim 1 , wherein the compensation operation comprises:
assigning a first weight to a first time series of the first data set during training of the recurrent neural network model; and
assigning a second weight to a second time series of the first data set during training of the recurrent neural network model.

5. The system as recited in claim 1 , wherein the compensation operation comprises applying a scaling function to at least one of: (a) an input of the recurrent neural network model for a particular time series, or (b) an output of the recurrent neural network model for the particular time series.

6. A method, comprising:
performing, by one or more computing devices:
obtaining an indication of a first data set comprising a respective time series of demand observations for a plurality of items;
training a recurrent neural network model to generate demand forecasts for a plurality of different items, wherein training of the recurrent neural network model is based at least in part on (a) analysis of a plurality of time series of the first data set and (b) performing one or more compensation operations to the plurality of times series of the first data set to compensate for statistical differences between respective distributions of the demand observations for different items within the first data set, wherein, within the training data set of the recurrent neural network model, a count of demand observations of at least one other item exceeds a count of demand observations of a target item;
generating respective probabilistic demand forecasts for different target items using one or more executions of the same recurrent neural network model; and
providing, via a programmatic interface, respective representations of the probabilistic demand forecasts.

7. The method as recited in claim 6 , wherein generating the probabilistic demand forecasts comprises:
predicting, using the recurrent neural network model, one or more parameters of a first demand distribution corresponding to a first time step; and
utilizing a sample from the first demand distribution as input to predict parameters of a second demand distribution corresponding to a second time step.

8. The method as recited in claim 6 , wherein a particular compensation operation comprises excluding a sample of the first data set from the training data set.

9. The method as recited in claim 6 , wherein a particular compensation operation of the one or more compensation operations comprises:
assigning a first weight to a first time series of the first data set during training of the recurrent neural network model; and
assigning a second weight to a second time series of the first data set during training of the recurrent neural network model.

10. The method as recited in claim 6 , wherein a particular compensation operation of the one or more compensation operations comprises:
applying a normalization function to at least one of: (a) an input of the recurrent neural network model, or (b) an output of the recurrent neural network model.

11. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
utilizing, as input for at least one execution of the one or more executions of the recurrent neural network model, an indication of a similarity between at least one of the target items and a second item, wherein, within the training data set, a count of demand observations of the second item exceeds the count of demand observations of the at least one target item.

12. The method as recited in claim 11 , wherein the indication of a similarity comprises one or more of: (a) a price, (b) a product category, (c) item introduction timing information, or (d) marketing information.

13. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
utilizing, as input for at least one execution of the one or more executions of the recurrent neural network model, a time series of demand of at least one of the target items.

14. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
selecting a first sub-sequence of elements of a first time series as a first training example for the recurrent neural network model; and
selecting a second sub-sequence of elements of the first time series as a second training example for the recurrent neural network model, wherein the first sub-sequence differs from the second sub-sequence in one or more of: (a) a starting offset within the first time series or (b) a number of elements.

15. The method as recited in claim 6 , wherein training the neural network model comprises utilizing a probabilistic sampling technique to determine whether input for a particular time step is to include (a) a sample from a demand distribution predicted by the recurrent neural network model for a previous time step or (b) a demand observation corresponding to the previous time step, wherein according to the probabilistic sampling technique, the probability of utilizing the sample increases as the total number of training iterations increases.

16. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors cause the one or mo

Assignees

• Amazon Tech Inc

Inventors

• Flunkert Valentin
• Salinas David Jean Bernard Alfred

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06Q30/0202 Primary

  Market predictions or forecasting for commercial activities · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title