Using facial recognition and facial expression detection to analyze in-store activity of a user

Patent metadata

Field	Value
Publication number	US-2017337602-A1
Application number	US-201615160910-A
Country	US
Kind code	A1
Filing date	May 20, 2016
Priority date	May 20, 2016
Publication date	Nov 23, 2017
Grant date	—

Abstract

One or more embodiments of the disclosure provide systems and methods providing providing improved customer service to visiting customers of a brick-and-mortar merchant location. For example, a customer recognition system intelligently detects and notifies a merchant when a customer is in need of assistance based on the customer's facial expression. The customer recognition system can also identify a product associated with the customer need. In some examples, the customer recognition system identifies a user profile associated with a customer shopping at a merchant location, determines a trust level for the customer based on user profile information, and based on the trust level, causes a secured product display to provide a customer access to a secured product.

First claim

What is claimed is:

1 . A method comprising:
receiving an image that portrays a customer and an indication that the customer desires to access a secured product within a secured product display at a physical location of the merchant; analyzing, by at least one processor, the image to identify the customer portrayed in the image; based on identifying the customer, accessing profile information associated with the customer to determine a trust level for the customer; and based on the trust level for the customer, sending a communication that allows the customer to gain access to the secured product within the secured product display at the physical location of the merchant.

2 . The method of claim 1 , wherein analyzing the image to identify the customer comprises comparing the image that portrays the customer with one or more user images that each correspond to a user profile.

3 . The method of claim 2 , further comprising identifying a subset of user profiles associated with the physical location of the merchant, and wherein the one or more user images correspond to the subset of user profiles.

4 . The method of claim 3 , further comprising:
receiving geo-location information indicating the location of a plurality of users associated with a corresponding plurality of user profiles; and wherein identifying the subset of user profiles comprises determining one or more user profiles from the plurality of user profiles associated with a user located within a defined proximity of the physical merchant location based on the geo-location information.

5 . The method of claim 2 , wherein comparing the image that portrays the customer with the one or more user images comprises using facial recognition techniques to identify a user image that matches the image portraying the customer.

6 . The method of claim 1 , further comprising determining the trust level for the customer is sufficient to grant the customer access to the secured product display based on one or more characteristics of the secured product.

7 . The method of claim 1 , wherein the communication to the merchant that allows the customer to gain access to the secured product causes the secured product display to unlock a display door that prevented the customer from accessing to the secured products.

8 . The method of claim 1 , further comprising generating a first access code that the customer provides at the secured product display, and wherein sending the communication to allow the customer to gain access to the secured product comprises sending the first access code to a client device corresponding to the customer.

9 . The method of claim 8 , further comprising generating a second access code that is paired with the first access code, and wherein sending the communication to allow the customer to gain access to the secured product further comprises providing the second access code to verify the first access code.

10 . The method of claim 1 , wherein the profile information associated with the customer to determine the trust level comprises previous transactions of the customer with the merchant and previous transactions of the customer with one or more additional merchants.

11 . A method comprising:
receiving an image that portrays a customer located at a physical merchant location; analyzing, by at least one processor, the image to identity a facial expression type of the customer portrayed in the image; based on the facial expression type, determining a potential customer need associated with the customer; and providing, to a client device associated with the merchant location, a notification of the potential customer need.

12 . The method of claim 11 , further comprising accessing a plurality of image groups each associated with a particular facial expression type.

13 . The method of claim 12 , wherein analyzing the image to identify the facial expression type of the customer portrayed in the image comprises determining an image group from the plurality of image groups that corresponds to facial expression features of the customer portrayed in the image.

14 . The method of claim 11 , further comprising, based on the facial expression type, that the potential customer need is a customer question regarding a product.

15 . The method of claim 14 , further comprising identifying the product corresponding to the potential customer need based on detecting the product within the image.

16 . The method of claim 14 , further comprising identifying the product corresponding to the potential customer need based on a camera location within the physical merchant location.

17 . The method of claim 11 , further comprising:
using facial detection techniques to determine the identity of the customer; and generating the notification to include user profile information associated with the identity of the customer.

18 . A system comprising:
at least one processor; and a non-transitory computer readable medium storing instructions that, when executed by the at least one processor, cause the system to:
receive an image that portrays a customer located at a physical merchant location;
analyze, by at least one processor, the image to identity a facial expression type of the customer portrayed in the image;
based on the facial expression type, determine a potential customer need associated with the customer; and
provide, to a client device associated with the merchant location, a notification of the potential customer need.

19 . The system of claim 18 , further comprising instructions that, when executed by the at least one processor, cause the system to access merchant employee profile information to identify one or more merchant employees available to respond to the potential customer need.

20 . The system of claim 19 , further comprising analyzing the merchant profile information to identify a particular merchant employee from the one or more available merchant employees that is most suited to respond to the potential customer need based on at least one characteristic of the customer need and at least one attribute within the merchant profile of the particular merchant employee.

Assignees

• Facebook Inc

Inventors

• Davis Stephen Moore

Classifications

• G06Q10/08

  Logistics, e.g. warehousing, loading or distribution; Inventory or stock management · CPC title

• G06Q30/0617

  Representative agent · CPC title

• H04W4/029

  Location-based management or tracking services · CPC title

• H04L67/306

  User profiles · CPC title

• G06Q30/0607 Primary

  Regulating the sale of restricted items, e.g. alcohol · CPC title