Methods and apparatus to generate textual data using machine learning processes

Patent metadata

Field	Value
Publication number	US-11636267-B2
Application number	US-202117162364-A
Country	US
Kind code	B2
Filing date	Jan 29, 2021
Priority date	Jan 29, 2021
Publication date	Apr 25, 2023
Grant date	Apr 25, 2023

Abstract

This application relates to apparatus and methods for automatically generating item information, such as item descriptions, and providing the item information to customers. For example, the embodiments may generate and provide personalized item descriptions to customers during conversational interactions in speech-based systems. In some examples, the embodiments determine entities (e.g., attributes) from item information, and apply trained machine learning processes to the extracted entities to generate textual data, such as item descriptions. For example, a computing device may apply a trained natural language processing, such as a trained transformer-based machine learning technique, to the extracted entities to generate the item descriptions. In some examples, the computing device applies post processing techniques to the generated textual data. The generated textual data may include descriptive phrases that are user friendly to customers in an e-commerce system. The textual data can be converted to audio and played to customers.

First claim

What is claimed is:

1. A system comprising:
a computing device configured to:
generate a sentence embedding, a token embedding, and a position embedding based on each of item titles for a plurality of items;
train a natural language processing model with the sentence embeddings, the token embeddings, and the position embeddings;
receive an item description request;
generate word embeddings based on the item description request;
determine at least one entity based on application of a neural network to the word embeddings;
generate an item description based on applying the trained natural language processing model to at least a portion of the word embeddings and the at least one entity; and
transmit the item description in response to the item description request.

2. The system of claim 1 , wherein determining at least one entity comprises:
applying a bi-directional long short term memory neural network model to the word embeddings to generate a context vector;
applying an attention model to the context vector to generate attention values; and
applying a conditional random field model to the attention values to determine sequence tags for the word embeddings.

3. The system of claim 2 , wherein the computing device is configured to:
determine attributes within the context vector based on the sequence tags; and
replace the attributes with tokens.

4. The system of claim 3 , wherein generating the item description comprises replacing the tokens with the corresponding attributes.

5. The system of claim 3 , wherein the trained natural language processing model is trained to keep the tokens.

6. The system of claim 1 , wherein the computing device is configured to randomly mask a portion of the item titles to train the natural language processing model.

7. The system of claim 1 , wherein the computing device is configured to obtain an item title from a data repository in response to the item description request, wherein the word embeddings are generated based on the item title.

8. The system of claim 1 , wherein the item request comprises textual data converted from audio.

9. A method comprising:
generating a sentence embedding, a token embedding, and a position embedding based on each of item titles for a plurality of items;
training a natural language processing model with the sentence embeddings, the token embeddings, and the position embeddings;
receiving an item description request;
generating word embeddings based on the item description request;
determining at least one entity based on application of a neural network to the word embeddings;
generating an item description based on applying the trained natural language processing model to at least a portion of the word embeddings and the at least one entity; and
transmitting the item description in response to the item description request.

10. The method of claim 9 , wherein determining at least one entity comprises:
applying a bi-directional long short term memory neural network model to the word embeddings to generate a context vector;
applying an attention model to the context vector to generate attention values; and
applying a conditional random field model to the attention values to determine sequence tags for the word embeddings.

11. The method of claim 10 , comprising:
determining attributes within the context vector based on the sequence tags; and
replacing the attributes with tokens, wherein the trained natural language processing model is trained to keep the tokens, and generating the item description comprises replacing the tokens with corresponding attributes.

12. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:
generating a sentence embedding, a token embedding, and a position embedding based on each of item titles for a plurality of items;
training a natural language processing model with the sentence embeddings, the token embeddings, and the position embeddings;
receiving an item description request;
generating word embeddings based on the item description request;
determining at least one entity based on application of a neural network to the word embeddings;
generating an item description based on applying the trained natural language processing model to at least a portion of the word embeddings and the at least one entity; and
transmitting the item description in response to the item description request.

13. The non-transitory computer readable medium of claim 12 , wherein determining at least one entity comprises:
applying a bi-directional long short term memory neural network model to the word embeddings to generate a context vector;
applying an attention model to the context vector to generate attention values; and
applying a conditional random field model to the attention values to determine sequence tags for the word embeddings.

14. The non-transitory computer readable medium of claim 13 further comprising instructions stored thereon that, when executed by at least one processor, further cause the device to perform operations comprising:
determining attributes within the context vector based on the sequence tags; and
replacing the attributes with tokens, wherein the trained natural language processing model is trained to keep the tokens, and generating the item description comprises replacing the tokens with corresponding attributes.

Assignees

• Walmart Apollo Llc

Inventors

• Kedia Shashank
• Mantha Aditya
• Guo Stephen Dean
• Achan Kannan

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F40/284 Primary

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title