Systems and methods for real-time document recommendation

Patent metadata

Field	Value
Publication number	US-12292906-B2
Application number	US-202318160449-A
Country	US
Kind code	B2
Filing date	Jan 27, 2023
Priority date	Jan 27, 2023
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

Embodiments described herein provide systems and methods for document recommendation. A system receives a set of training data including a plurality of documents. The system determines whether the set of training data includes annotated contextual information corresponding to the plurality of documents. The system trains supervised and/or unsupervised models based on the availability of data. The models are used to generate vectors representing the documents. During a live text conversation, text from the conversation may be vectorized using the models and the vectors compared to those representing the documents in order to find the most relevant documents. The system may generate an indication of a recommended document.

First claim

What is claimed is:

1. A method of document recommendation, the method comprising:
receiving, via a data interface, a set of training data including a plurality of documents;
determining whether the set of training data includes annotated contextual information corresponding to the plurality of documents;
training a document recommendation model using the plurality of documents according to an unsupervised procedure thereby producing a first plurality of vectors respectively representing the plurality of documents;
training the document recommendation model using the plurality of documents, and the annotated contextual information as supervisory labels, thereby producing a second plurality of vectors respectively representing the plurality of documents, when the set of training data includes the annotated contextual information;
receiving a user utterance associated with a conversation;
encoding the user utterance and/or at least part of the conversation into a conversation representation;
determining a plurality of scores by comparing the conversation representation with the first plurality or the second plurality of vectors; and
generating, by a decoder of the document recommendation model, an indication of a recommended document from the plurality of documents based on the plurality of scores.

2. The method of claim 1 , further comprising:
determining a document recommendation based on a first comparison between the representation of the utterance and the first plurality of vectors, and further based on a comparison between the representation of the utterance and the second plurality of vectors,
wherein the indication of the recommended document is based on the document recommendation.

3. The method of claim 1 , wherein the training the document recommendation model using the annotated contextual information is performed in response to a determination that an amount of annotated contextual information is above a predetermined threshold.

4. The method of claim 3 , wherein individual items of the annotated contextual information are included in the determined amount if they are in a selected natural language.

5. The method of claim 1 , further comprising:
training the document recommendation model using the plurality of documents, and the annotated contextual information as supervisory labels a second time in response to a determination that additional annotated contextual information is available.

6. The method of claim 1 , wherein the generating the indication of the recommended document is further based on respective popularity scores associated with the plurality of documents, wherein the respective popularity scores are based on a frequency with which each document of the plurality of documents is accessed.

7. The method of claim 1 , wherein the determining the plurality of scores comprises computing each respective score of the plurality of scores by a weighted sum of a first score associated with the first plurality of vectors and a second score associated with the second plurality of vectors.

8. A system for document recommendation, the system comprising:
a memory that stores a document recommendation model and a plurality of processor executable instructions;
a communication interface that receives a set of training data including a plurality of documents; and
one or more hardware processors that read and execute the plurality of processor-executable instructions from the memory to perform operations comprising:
determining whether the set of training data includes annotated contextual information corresponding to the plurality of documents;
training the document recommendation model using the plurality of documents according to an unsupervised procedure thereby producing a first plurality of vectors respectively representing the plurality of documents;
training the document recommendation model using the plurality of documents, and the annotated contextual information as supervisory labels, thereby producing a second plurality of vectors respectively representing the plurality of documents, when the set of training data includes the annotated contextual information;
receiving a user utterance associated with a conversation;
the user utterance and/or at least part of the conversation into a conversation representation;
determining a plurality of scores by comparing the conversation representation with the first plurality or the second plurality of vectors; and
generating, by a decoder of the document recommendation model, an indication of a recommended document from the plurality of documents based on the plurality of scores.

9. The system of claim 8 , the operations further comprising:
determining a document recommendation based on a first comparison between the representation of the utterance and the first plurality of vectors, and further based on a comparison between the representation of the utterance and the second plurality of vectors,
wherein the indication of the recommended document is based on the document recommendation.

10. The system of claim 8 , wherein the training the document recommendation model using the annotated contextual information is performed in response to a determination that an amount of annotated contextual information is above a predetermined threshold.

11. The system of claim 10 , wherein individual items of the annotated contextual information are included in the determined amount if they are in a selected natural language.

12. The system of claim 8 , the operations further comprising:
training the document recommendation model using the plurality of documents, and the annotated contextual information as supervisory labels a second time in response to a determination that additional annotated contextual information is available.

13. The system of claim 8 , wherein the generating the indication of the recommended document is further based on respective popularity scores associated with the plurality of documents, wherein the respective popularity scores are based on a frequency with which each document of the plurality of documents is accessed.

14. The system of claim 8 , wherein the determining the plurality of scores comprises computing each respective score of the plurality of scores by a weighted sum of a first score associated with the first plurality of vectors and a second score associated with the second plurality of vectors.

15. A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising:
receiving, via a data interface, a set of training data including a plurality of documents;
determining whether the set of training data includes annotated contextual information corresponding to the plurality of documents;
training a document recommendation model using the plurality of documents according to an unsupervised procedure thereby producing a first plurality of vectors respectively representing the plurality of documents;
training the document recommendation model using the plurality of documents, and the annotated contextual information as supervisory labels, thereby producing a second plurality of vectors respectively representing the plurality of documents, when the set of training data includes the annotated contextual information;
receiving a user utterance associated with a conversation;
encoding the user utterance and/or at least part of the conversation into a conversation representation;
determining a plurality of scores by comparing the conversation representa

Assignees

• Salesforce Inc

Inventors

• Jiang Feifei
• Kale Aron
• Kale Anuprit
• Asur Sitaram
• Cheng Na
• Alexander Zachary
• Yee Victor
• Ordaz Fermin

Classifications

• G06F16/3347

  using vector based model · CPC title

• G06F40/263

  Language identification · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F16/3329 Primary

  Natural language query formulation · CPC title