Parallel retrieval of training data from multiple producers for machine learning systems

Patent metadata

Field	Value
Publication number	US-2016019469-A1
Application number	US-201414335731-A
Country	US
Kind code	A1
Filing date	Jul 18, 2014
Priority date	Jul 18, 2014
Publication date	Jan 21, 2016
Grant date	—

Abstract

A sorting engine is an intermediary layer between a multi-threaded engine that obtains batches of training data from the producers in parallel and the underlying machine learning engine. The sorting engine includes a shared buffer that has various slots for storing batches of training data, where the slots are organized in a deterministic order associated with the producers. A batch of training data obtained by a thread from a given producer may be stored only in a corresponding slot in the shared buffer. Further, the sorting engine transmits the batch to the machine learning engine only when a previous batch in the deterministic order has been transmitted from the shared buffer to the machine learning engine.

First claim

What is claimed is:

1 . A method comprising:
providing a shared buffer for storing batches of training data obtained from a plurality of training data producers, the shared buffer configured according to a deterministic order of the plurality of training data producers; receiving a plurality of batches of training data obtained from the plurality of training data producers in parallel; storing the plurality of batches in the shared buffer in the deterministic order, the deterministic order being independent from an order in which the plurality of batches is received; and transmitting the plurality of batches to a training data consumer in the deterministic order.

2 . The method of claim 1 , wherein the shared buffer includes a different set of slots associated with each of the plurality of training data producers, the sets of slots organized according to the deterministic order.

3 . The method of claim 2 , wherein storing the plurality of batches in the shared buffer comprises:
for a first batch, identifying a first training data producer from which the batch was obtained; identifying a first set of slots in the shared buffer associated with the training data producer; and storing the first batch in a given slot of the first set of slots.

4 . The method of claim 3 , further comprising determining that the given slot corresponds to the first batch based on a batch index associated with the first batch, the batch index indicating a location within the first training data producer from which the first batch was obtained.

5 . The method of claim 1 , wherein each of the plurality of batches is obtained by a different thread executing in a multi-threaded execution environment.

6 . The method of claim 5 , wherein a first batch of the plurality of batches is obtained from a first data producer by a first thread, and further comprising:
before transmitting the first batch to the training data consumer, receiving a second batch obtained from the first producer by the first thread; and blocking the first thread until the first batch is transmitted to the training data consumer.

7 . The method of claim 6 , wherein blocking the first thread comprises:
determining that a slot in the shared buffer where the second batch should be stored is occupied by the first batch.

8 . The method of claim 2 , further comprising:
determining that no more batches are to be obtained from a first training data producer in the plurality of training data producers; and storing a dummy batch a first slot included in a first set of slots associated with the first training data producer.

9 . The method of claim 1 , wherein the training data consumer is a machine learning system configured to process the plurality of batches to identify one or more statistical similarities between data included in the plurality of batches.

10 . A computer-readable storage medium containing computer program code for:
providing a shared buffer for storing batches of training data obtained from a plurality of training data producers, the shared buffer configured according to a deterministic order of the plurality of training data producers; receiving a plurality of batches of training data obtained from the plurality of training data producers in parallel; storing the plurality of batches in the shared buffer in the deterministic order, the deterministic order being independent from an order in which the plurality of batches is received; and transmitting the plurality of batches to a training data consumer in the deterministic order.

11 . The computer-readable storage medium of claim 10 , wherein the shared buffer includes a different set of slots associated with each of the plurality of training data producers, the sets of slots organized according to the deterministic order.

12 . The computer-readable storage medium of claim 11 , wherein storing the plurality of batches in the shared buffer comprises:
for a first batch, identifying a first training data producer from which the batch was obtained; identifying a first set of slots in the shared buffer associated with the training data producer; and storing the first batch in a given slot of the first set of slots.

13 . The computer-readable storage medium of claim 12 , further comprising determining that the given slot corresponds to the first batch based on a batch index associated with the first batch, the batch index indicating a location within the first training data producer from which the first batch was obtained.

14 . The computer-readable storage medium of claim 10 , wherein each of the plurality of batches is obtained by a different thread executing in a multi-threaded execution environment.

15 . The computer-readable storage medium of claim 14 , wherein a first batch of the plurality of batches is obtained from a first data producer by a first thread, and further comprising:
before transmitting the first batch to the training data consumer, receiving a second batch obtained from the first producer by the first thread; and blocking the first thread until the first batch is transmitted to the training data consumer.

16 . The computer-readable storage medium of claim 15 , wherein blocking the first thread comprises:
determining that a slot in the shared buffer where the second batch should be stored is occupied by the first batch.

17 . The computer-readable storage medium of claim 11 , further comprising:
determining that no more batches are to be obtained from a first training data producer in the plurality of training data producers; and storing a dummy batch a first slot included in a first set of slots associated with the first training data producer.

18 . The computer-readable storage medium of claim 10 , wherein the training data consumer is a machine learning system configured to process the plurality of batches to identify one or more statistical similarities between data included in the plurality of batches.

19 . A computer system, comprising:
a plurality of training data producers configured to produce training data; a shared buffer for storing batches of training data obtained from a plurality of training data producers, the shared buffer configured according to a deterministic order of the plurality of training data producers; and a training data storing engine configured to:
receive a plurality of batches of training data obtained from the plurality of training data producers in parallel;
store the plurality of batches in the shared buffer in the deterministic order, the deterministic order being independent from an order in which the plurality of batches is received; and
transmit the plurality of batches to a training data consumer in the deterministic order.

20 . The system of claim 19 , wherein the shared buffer includes a different set of slots associated with each of the plurality of training data producers, the sets of slots organized according to the deterministic order.

Assignees

• Facebook Inc

Inventors

• Petrov Alexander Viktorovich

Classifications

• G06N99/005 Primary

  Physics · mapped topic

• G06N20/00 Primary

  Machine learning · CPC title

• G06F16/254

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title

• G06F9/46

  Multiprogramming arrangements · CPC title

• G06F9/52

  Program synchronisation; Mutual exclusion, e.g. by means of semaphores · CPC title