Voice-assisted scanning

Patent metadata

Field	Value
Publication number	US-9767501-B1
Application number	US-201314074346-A
Country	US
Kind code	B1
Filing date	Nov 7, 2013
Priority date	Nov 7, 2013
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

In some cases, a handheld device that includes a microphone and a scanner may be used for voice-assisted scanning. For example, a user may provide a voice input via the microphone and may activate the scanner to scan an item identifier (e.g., a barcode). The handheld device may communicate voice data and item identifier information to a remote system for voice-assisted scanning. The remote system may perform automatic speech recognition (ASR) operations on the voice data and may perform item identification operations based on the scanned identifier. Natural language understanding (NLU) processing may be improved by combining ASR information with item information obtained based on the scanned identifier. An action may be executed based on the likely user intent.

First claim

What is claimed is:

1. A system comprising:
one or more processors;
memory; and
one or more computer-executable instructions stored in the memory and executable by the one or more processors to:
receive, from a handheld electronic device, voice data and item identifier information, wherein the handheld electronic device includes at least a microphone to receive a voice input from a user and a scanner to scan an identifier of an item;
determine, based at least in part on the item identifier information, information about the item;
generate one or more transcriptions of the voice data using a speech recognition model;
generate a semantic representation of the one or more transcriptions using a natural language understanding model;
identify a reference to the item in the semantic representation;
identify a user intent in the semantic representation;
determine an action based at least in part on the information about the item, the reference to the item in the semantic representation, and the user intent in the semantic representation; and
perform the action.

2. The system as recited in claim 1 , wherein the action includes at least one of:
adding a particular quantity of the item to a virtual cart of the user;
removing the item from the virtual cart of the user;
communicating, to the handheld electronic device, product information about the item; or
communicating, to the handheld electronic device, product information associated with another item that is identified as being related to the item based on one or more properties of the item.

3. The system as recited in claim 1 , further comprising:
an item database communicatively coupled to at least one of the memory or the one or more processors, the item database including information associated with a plurality of items, each item of the plurality of items being associated with an individual barcode,
wherein the item identifier information comprises a barcode and the one or more computer-executable instructions are further executable by the one or more processors to query the item database based on the barcode to identify the information about the item.

4. The system as recited in claim 1 , wherein the reference to the item comprises an anaphora.

5. The system as recited in claim 1 , wherein at least one of the speech recognition model or the natural language understanding model was created using at least a portion of the information about the item.

6. A computer-implemented method comprising:
receiving, via a network, voice data and an identifier of an item, wherein the identifier of the item was obtained by scanning the item;
determining, based at least in part on the identifier of the item, information about the item;
generating a semantic representation of the voice data using at least one of a speech recognition model or a natural language understanding model;
identifying a reference to the item in the semantic representation;
identifying a user intent in the semantic representation; and
performing an action that is determined based at least in part on the information about the item, the reference to the item in the semantic representation, and the user intent in the semantic representation.

7. The computer-implemented method as recited in claim 6 , wherein the reference to the item comprises an anaphora.

8. The computer-implemented method as recited in claim 6 , wherein the speech recognition model is obtained using the information about the item.

9. The computer-implemented method as recited in claim 6 , wherein the natural language understanding model is obtained using the information about the item.

10. The computer-implemented method as recited in claim 6 , further comprising determining the identifier of the item based at least partly on a barcode, a quick response (QR) code, a radio-frequency identification (RFID), a near-field communication (NFC) identifier, a product logo, or an image on a product package of the item.

11. The computer-implemented method as recited in claim 6 , wherein performing the action comprises:
querying an item database to determine product information associated with the item; and
initiating communication of the product information associated with the item via the network.

12. The computer-implemented method as recited in claim 6 , wherein performing the action comprises:
identifying second item that is related to the item based on one or more properties of the item;
determining product information associated with the second item; and
initiating communication of the product information associated with the second item via the network.

13. The computer-implemented method as recited in claim 6 , wherein the user intent comprises an indication of a quantity of the item, a modification of a characteristic of the item, a request for the information about the item, or an indication to add the item to a virtual shopping cart.

14. The computer-implemented method as recited in claim 6 , wherein the semantic representation comprises one or more named entities.

15. A method comprising:
receiving, from a handheld electronic device, voice data associated with a voice input from a user and an identifier of an item, wherein the identifier of the item was obtained by scanning the item;
determining, based at least in part on the identifier of the item, information about the item;
generating a semantic representation of the voice data using at least one of a speech recognition model or a natural language understanding model;
identifying a reference to the item in the semantic representation;
identifying a user intent in the semantic representation; and
performing an action that is determined based at least in part on the information about the item, the reference to the item in the semantic representation, and the user intent in the semantic representation.

16. The method as recited in claim 15 , wherein the reference to the item comprises an anaphora.

17. The method as recited in claim 15 , wherein the identifier of the item is scanned prior to receiving the voice input from the user, after receiving the voice input from the user, or while at least a portion of the voice input is being received from the user.

18. The method as recited in claim 15 , wherein the voice data includes a recording of the voice input from the user that is stored as an audio file in a memory of the handheld electronic device.

19. The method as recited in claim 15 , wherein performing the action comprises initiating communication of information to be presented in a visual format via a user interface of the handheld electronic device.

20. The method as recited in claim 15 , wherein performing the action comprises initiating communication of information to be presented in an audible format via a speaker of the handheld electronic device.

21. The computer-implemented method as recited in claim 6 , wherein scanning of the item is performed by a scanner that scans at least one of a barcode, a QR code, an RFID, or an NFC identifier associated with the item.

Assignees

• Amazon Tech Inc

Inventors

• Schaaf Thomas
• Salvador Stan Weidner

Classifications

• G06Q30/0623 Primary

  by investigating goods or services · CPC title

• G06F17/20

  Physics · mapped topic

• G06F40/00

  Handling natural language data (speech analysis or synthesis, speech recognition G10L) · CPC title