Method and apparatus for recommending customer item based on visual information

Patent metadata

Field	Value
Publication number	US-11188965-B2
Application number	US-201816161153-A
Country	US
Kind code	B2
Filing date	Oct 16, 2018
Priority date	Dec 29, 2017
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

An apparatus for recommending a customer item identifies a purchase tendency of a customer based on an image, determines a recommended item for the customer by selecting a purchase tendency model corresponding to the purchase tendency, and provides information associated with the determined recommended item.

First claim

What is claimed is:

1. A processor-implemented method of recommending a customer item, the method comprising:
acquiring, using a camera, an image corresponding to a customer;
generating a segmented image by segmenting the acquired image using a segmentation model including one or more neural networks, where the segmented image indicates a category of one or more pixels of the image corresponding to the customer;
extracting a partial image feature vector of the customer from the segmented image;
calculating, for each of one or more purchase tendency models of a purchase tendency model database, a correlation level between the customer and the purchase tendency model based on a similarity between the partial image feature vector and a feature vector determined for the purchase tendency model;
selecting, from the purchase tendency models, a purchase tendency model having a highest correlation level among the correlation levels;
determining a recommended item for the customer based on the selected purchase tendency model; and
providing, using a display, information associated with the recommended item.

2. The method of claim 1 , wherein the determining comprises:
identifying a purchase tendency of the customer based on the acquired image; and
determining the recommended item for the customer based on the purchase tendency.

3. The method of claim 1 , wherein the acquiring of the image comprises acquiring at least one customer image using the camera, and wherein a field of view (FOV) of the camera covers at least a portion of an area in a physical store.

4. The method of claim 1 , wherein the acquiring of the image comprises acquiring the image with respect to the customer from among customers shown in a user terminal.

5. The method of claim 1 ,
wherein each of the purchase tendency models comprise a feature vector of an item corresponding to a fashion category, a feature vector of the acquired image, and a feature vector of an item corresponding to a non-fashion category,
wherein the calculating comprises calculating the correlation level between the customer and each of the purchase tendency models based on the feature vector of the item corresponding to the fashion category, the feature vector of the acquired image, the feature vector of the item corresponding to the non-fashion category, and the image feature of the customer.

6. The method of claim 1 , further comprising:
building the purchase tendency model database by building a purchase tendency model using an online purchase database.

7. The method of claim 1 , further comprising:
building the purchase tendency model database by building an offline purchase tendency model using a physical store purchase database.

8. The method of claim 1 , wherein the determining of the recommended item based on the selected purchase tendency model comprises:
matching an item image of a corresponding item and the acquired image, with an item corresponding to each category in the selected purchase tendency model; and
determining the recommended item based on a fitting score calculated between the item image and the acquired image.

9. The method of claim 1 , wherein the providing comprises:
transmitting a push request to a customer terminal including an augmented reality (AR) display; and
receiving information indicating whether the push request is accepted from the customer terminal.

10. The method of claim 1 , wherein the providing comprises transmitting the information associated with the recommended item to a customer terminal in response to a push request being accepted by the customer terminal.

11. The method of claim 1 , wherein the providing comprises transmitting item information related to the recommended item to a display disposed within a threshold distance from a position of the customer.

12. The method of claim 1 , wherein the providing comprises transmitting the information associated with the recommended item to any one or any combination of a work terminal and a customer terminal.

13. The method of claim 1 , wherein the determining comprises determining the recommended item based on a purchase area in which the customer is positioned and the acquired image.

14. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

15. A processor-implemented method of recommending a customer item, the method comprising:
acquiring, using a camera, an image corresponding to a customer;
generating a segmented image by segmenting the acquired image using a segmentation model including one or more neural networks, where the segmented image indicates a category of one or more pixels of the image corresponding to the customer;
extracting an image feature of the customer from the segmented image;
retrieving a purchase tendency model corresponding to the customer from a purchase tendency model database based on the image feature;
determining a recommended item for the customer based on the purchase tendency model corresponding to the customer;
providing, using a display, information associated with the recommended item; and
building the purchase tendency model database by building the purchase tendency model using an online purchase database, wherein the building comprises:
training the purchase tendency model with statistics of online purchase records of the customer;
determining a feature vector of the customer from decomposing a purchase history matrix indicating an item purchase history of the customer;
classifying customers into a plurality of customer clusters based on the feature vector of the customer; and
determining a feature vector of an item corresponding to a category based on per-category purchase data regarding each of the plurality of customer clusters.

16. A processor-implemented method of recommending a customer item, the method comprising:
acquiring, using a camera, an image corresponding to a customer;
generating a segmented image by segmenting the acquired image using a segmentation model including one or more neural networks, where the segmented image indicates a category of one or more pixels of the image corresponding to the customer;
extracting an image feature of the customer from the segmented image;
calculating, based on the image feature, a correlation level between the customer and each of purchase tendency models of a purchase tendency model database;
selecting, from the purchase tendency models, a purchase tendency model having a highest correlation level among the correlation levels;
determining a recommended item for the customer based on the selected purchase tendency model;
providing, using a display, information associated with the recommended item; and
building the purchase tendency model database by building an offline purchase tendency model using a physical store purchase database, wherein the building comprises:
extracting a customer identification (ID), the acquired image, and purchase records of the customer from the physical store purchase database;
obtaining appearance information of the customer and purchase action information of the customer by analyzing the acquired image;
classifying customers into a plurality of customer clusters based on the information obtained from the acquired image; and
calculating a feature vector of an item corresponding to a category based on per-category purchase data regarding each of the plurality of customer clusters.

17. A processor-implemented method of recommending a customer item, the method comprising:
a

Assignees

• Samsung Electronics Co Ltd

Inventors

• Li Weiming
• Yu Dongdong
• Liu Yang
• Wang Qiang
• Ahn Minsu
• Hong Sunghoon
• Kao Yueying
• Wang Zairan
• Wang Hao

Classifications

• G06Q30/0631 Primary

  Recommending goods or services · CPC title

• G06F16/583

  using metadata automatically derived from the content · CPC title

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06Q30/0643

  graphically representing goods, e.g. 3D product representation · CPC title