Indirect feedback systems and methods

Patent metadata

Field	Value
Publication number	US-10019489-B1
Application number	US-201615140341-A
Country	US
Kind code	B1
Filing date	Apr 27, 2016
Priority date	Apr 27, 2016
Publication date	Jul 10, 2018
Grant date	Jul 10, 2018

Abstract

Features are disclosed for identifying indirect user feedback and providing content such as item descriptions based on the indirect user feedback. An indirect feedback system may receive sensed data such as images or audio and identify indicators of indirect feedback for the subject shown, heard, or otherwise detected in the sensed data. For example, a user's facial expression and/or body language can provide indirect feedback as to how the user is feeling (e.g., mood). Based on the detected mood, features for suggesting additional items that should appeal to the user are described.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
under control of a computing device executing specific computer-executable instructions,
capturing, via an imaging device, an image including a subject;
identifying, using pixel information of pixels included in the image, a portion of the image including the subject;
identifying a facial feature of the subject included in the portion of the image;
detecting a mood of the subject at a time that the image was captured based at least in part on the facial feature;
identifying, using pixel information of pixels included in the portion of the image, a first configuration of an item shown in the portion of the image including the subject;
identifying a second configuration of the item using the mood and the first configuration, wherein the first configuration identifies a first color for the item and the second configuration identifies a second color for the item; and
causing an image of the item in the second configuration to be presented on a display.

2. The computer-implemented method of claim 1 , further comprising:
generating additional mood data for the subject using at least a portion of an additional image including the subject, the additional mood data indicating an additional detected mood of the subject at a time that the additional image was captured;
identifying, using pixel information of pixels included in the portion of the additional image, the first configuration of the item shown in the portion of the additional image including the subject;
identifying a time difference between the time that the image was captured and the time that the additional image was captured; and
generating a weighted detected mood using the mood, the additional mood data, and the time difference,
wherein identifying the second configuration of the item further uses the weighted detected mood.

3. The computer-implemented method of claim 1 , wherein identifying the second configuration of the item comprises:
transmitting, to an item data store, a search request for an item associated with the second configuration; and
receiving, from the item data store, search results including the second configuration of the item.

4. The computer-implemented method of claim 3 , further comprising:
capturing, via the imaging device, a second image including the subject after the presentation of the second configuration of the item;
identifying, using pixel information of pixels included the second image, a portion of the second image including the subject;
identifying a second facial feature of the subject including in the portion of the image;
detecting a second mood of the subject at a time that the second image was captured based at least in part on the second facial feature;
determining the mood indicates a first level of happiness greater than a second level of happiness indicated by the second mood;
identifying a third configuration of the item using the first configuration of the item and the second configuration of the item, wherein the third configuration identifies a third color for the item; and
causing an image of the item in the third configuration to be presented on the display.

5. A computer-implemented method comprising:
under control of a computing device executing specific computer-executable instructions,
receiving, from a data capture device, data related to an item in a first configuration and a user;
identifying the first configuration of the item;
detecting, from the data related to the item in the first configuration and the user, indirect feedback of the user, wherein the indirect feedback is related to the item in the first configuration;
identifying a second configuration of the item using the indirect feedback and the first configuration of the item, wherein the first configuration of the item is different than the second configuration of the item; and
causing an image of the item in the second configuration to be presented on a display.

6. The computer-implemented method of claim 5 , wherein the data capture device comprises a microphone and the data related to the item comprises audio data representing an utterance of the user.

7. The computer-implemented method of claim 6 , wherein the indirect feedback comprises a mood of the user and wherein the mood of the user is detected from at least one of:
an acoustic property of the audio data; and
a word included in the utterance of the user.

8. The computer-implemented method of claim 7 , further comprising:
transmitting the audio data to an automatic speech recognition system; and
receiving a transcript of the audio data from the automatic speech recognition system, wherein the transcript includes the word.

9. The computer-implemented method of claim 5 , wherein the data capture device comprises an imaging device and the data related to the item in the first configuration comprises an image of the user and the item in the first configuration.

10. The computer-implemented method of claim 9 , wherein the indirect feedback of the user comprises a mood of the user, and wherein the computer-implemented method further comprises applying a mood detection model to pixel information from the image to detect the mood of the user.

11. The computer-implemented method of claim 9 , wherein the indirect feedback of the user comprises a mood of the user, and wherein the mood of the user is detected from at least one of:
a facial expression of the user included in the image; and
a posture of the user included in the image.

12. The computer-implemented method of claim 9 , wherein the indirect feedback of the user comprises a first mood of the user, and wherein the computer-implemented method further comprises:
receiving another image of the user and the item in the first configuration from the imaging device;
detecting, from the other image of the user and the item in the first configuration, a second mood of the user related to the item in the first configuration;
generating a weighted mood from the first mood and the second mood; and
wherein identifying the second configuration of the item further uses the weighted mood.

13. The computer-implemented method of claim 9 , wherein:
the indirect feedback of the user comprises a first mood of the user,
the image includes another user,
the computer-implemented method further comprises detecting a second mood of the other user included in the image, and
identifying the second configuration of the item further uses the second mood of the other user included in the image.

14. The computer-implemented method of claim 5 , wherein identifying the first configuration of the item comprises:
receiving a first image showing the item in the first configuration;
identifying pixels in the first image showing the item; and
identifying the first configuration of the item using the pixels in the first image showing the item.

15. The computer-implemented method of claim 5 , wherein identifying the first configuration of the item comprises receiving a radio frequency signal including an identifier for the first configuration of the item.

16. A system comprising:
a data store storing specific computer-executable instructions; and
a computing device in communication with the data store, the computing device including a processor that executes the specific computer-executable instructions to cause the system to at least:
receive, from a data capture device, data related to an item in a first configuration and a user;
identify the first configuration of the item;
detect, from the data related to t

Assignees

• Amazon Tech Inc

Inventors

• Adair Brian Lafayette
• Levy Joshua Seth

Classifications

• G06F17/3053 Primary

  Physics · mapped topic

• G06Q30/0631 Primary

  Recommending goods or services · CPC title

• G06K9/00302

  Physics · mapped topic

• G06F17/3028

  Physics · mapped topic

• G06K9/00288

  Physics · mapped topic