Method and system for managing embeddings and serving recommendations in real-time

Patent metadata

Field	Value
Publication number	US-12430583-B2
Application number	US-202117513099-A
Country	US
Kind code	B2
Filing date	Oct 28, 2021
Priority date	May 14, 2021
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

The disclosed embodiments describe techniques for isolating and managing models via versioning. In one embodiment, a method is disclosed comprising reading a configuration document associated with a first model; incrementing an internal write version of the configuration document; storing the internal write version in the configuration document; generating documents belonging to a second model such that respective document identifiers of the documents include a next external version a first field and the incremented internal write version in a second field; and uploading the documents to a serving system, causing the serving system to replace the first model with the second model.

First claim

We claim:

1. A method comprising:
reading, by a processing device, a configuration document associated with a first model;
incrementing, by the processing device, an internal write version of the configuration document;
storing, by the processing device, the internal write version in the configuration document;
generating, by the processing device, documents belonging to a second model such that respective document identifiers of the documents include a next external version in a first field and the incremented internal write version in a second field, the first field and second field comprising distinct sections of the document identifiers that maintain version coherence for model-specific embeddings during model transitions; and
uploading, by the processing device, the documents to a serving system, causing the serving system to atomically replace the first model with the second model, the atomic replacement ensuring that inferences access embeddings belonging to a single model version.

2. The method of claim 1 , further comprising:
updating the configuration document such that an internal read version of the configuration document stores the incremented internal write version and an external version of the configuration document stores a next value.

3. The method of claim 2 , further comprising:
reading the configuration document to obtain the internal read version; and
filtering accesses to the documents during an inference by the serving system such that only documents storing the read version are utilized.

4. The method of claim 1 , wherein the documents of the second model include model-related data, the model-related data comprising updated embeddings and shared documents associated with the second model include model-agnostic data.

5. The method of claim 1 , further comprising:
detecting and discarding obsolete documents by:
separating the documents into discrete sets so that a first subset documents are automatically overwritten in each model update and are not be explicitly discarded; and
detecting a second subset documents that have not been overwritten by comparing an associated internal read version to a current internal read version in the configuration document.

6. The method of claim 1 , further comprising cycling the external version of the document identifiers through a range of numbers.

7. The method of claim 6 , wherein the range of numbers comprises 0 and 1.

8. A non-transitory computer-readable storage medium for tangibly storing computer program instructions capable of being executed by a computer processor, the computer program instructions defining steps of:
reading a configuration document associated with a first model;
incrementing an internal write version of the configuration document;
storing the internal write version in the configuration document;
generating documents belonging to a second model such that respective document identifiers of the documents include a next external version a first field and the incremented internal write version in a second field, the first field and second field comprising distinct sections of the document identifiers that maintain version coherence for model-specific embeddings during model transitions; and
uploading the documents to a serving system, causing the serving system to atomically replace the first model with the second model, the atomic replacement ensuring that inferences access embeddings belonging to a single model version.

9. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:
updating the configuration document such that an internal read version of the configuration document stores the incremented internal write version and an external version of the configuration document stores a next value.

10. The non-transitory computer-readable storage medium of claim 8 , wherein the documents of the second model include model-related data, the model-related data comprising updated embeddings and shared documents associated with the second model include model-agnostic data.

11. The non-transitory computer-readable storage medium of claim 10 , the steps further comprising:
reading the configuration document to obtain the internal read version; and
filtering accesses to the documents during an inference by the serving system such that only documents storing the read version are utilized.

12. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising:
detecting and discarding obsolete documents by:
separating the documents into discrete sets so that a first subset documents are automatically overwritten in each model update and are not be explicitly discarded; and
detecting a second subset documents that have not been overwritten by comparing an associated internal read version to a current internal read version in the configuration document.

13. The non-transitory computer-readable storage medium of claim 8 , the steps further comprising cycling the external version of the document identifiers through a range of numbers.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the range of numbers comprises 0 and 1.

15. A device comprising:
a processor configured to:
read a configuration document associated with a first model;
increment an internal write version of the configuration document;
store the internal write version in the configuration document;
generate documents belonging to a second model such that respective document identifiers of the documents include a next external version a first field and the incremented internal write version in a second field, the first field and second field comprising distinct sections of the document identifiers that maintain version coherence for model-specific embeddings during model transitions; and
upload the documents to a serving system, causing the serving system to atomically replace the first model with the second model, the atomic replacement ensuring that inferences access embeddings belonging to a single model version.

16. The device of claim 15 , the processor further configured to:
update the configuration document such that an internal read version of the configuration document stores the incremented internal write version and an external version of the configuration document stores a next value.

17. The device of claim 15 , wherein the documents of the second model include model-related data, the model-related data comprising updated embeddings and shared documents associated with the second model include model-agnostic data.

18. The device of claim 17 , the processor further configured to:
read the configuration document to obtain the internal read version; and
filter accesses to the documents during an inference by the serving system such that only documents storing the read version are utilized.

19. The device of claim 15 , the processor further configured to:
detect and discard obsolete documents by:
separating the documents into discrete sets so that a first subset documents are automatically overwritten in each model update and are not be explicitly discarded; and
detecting a second subset documents that have not been overwritten by comparing an associated internal read version to a current internal read version in the configuration document.

20. The device of claim 15 , the processor further configured to cycle the external version of the document identifiers through a range of numbers.

Assignees

• Yahoo Ad Tech Llc

Inventors

• Gilad Eran
• Keidar Idit
• Hillel Eshcar
• Basin Dmitry

Classifications

• G06F40/00

  Handling natural language data (speech analysis or synthesis, speech recognition G10L) · CPC title

• G06F40/197

  Version control (for software G06F8/71) · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N20/00 Primary

  Machine learning · CPC title