Image display device, image display system, image display method, and program

Patent metadata

Field	Value
Publication number	US-12167803-B2
Application number	US-202318237240-A
Country	US
Kind code	B2
Filing date	Aug 23, 2023
Priority date	Mar 22, 2016
Publication date	Dec 17, 2024
Grant date	Dec 17, 2024

Abstract

An image display device includes an approach situation information acquirer that acquires approach situation information indicating approach of at least part of a customer's body to a shelf on which a plurality of items are displayed and indicating, when there is the approach, a position of the approach on the shelf at each time; an associator that associates the approach situation information with one item of the plurality of items according to the position indicated by the approach situation information; an index value calculator that calculates an index value relating to the customer's behavior on the basis of information indicating the item associated with the approach situation information; and a display unit that displays the index value calculated by the index value calculator together with an image of the item on the basis of the association made by the associator.

First claim

The invention claimed is:

1. An information processing device comprising:
at least one memory configured to store instructions; and
at least one processor configured to execute the instructions to:
receive an operation for designating a first product by a user among a plurality of products displayed on a shelf;
determine, based on time series of images capturing the shelf, a second product touched by a customer who touched the first product;
receive sensing data representing movement of a customer's hand toward the shelf;
determine, based on the time series of the images and the sensing data, the second product touched by the customer who touched the first product;
measure a distance between the customer's hand and a sensor to determine the second product touched by the customer who touched the first product based on the distance; and
display, on a display, a shelf image including an image of the plurality of the products,

wherein the shelf image includes the first product and the second product.

2. The information processing device according to claim 1 , wherein the shelf image is a second shelf image,
wherein the processor is further configured to execute the instructions to:
display, on the display, a first shelf image including an image of the plurality of the products; and
receive, the operation for designating the first product by the user with the image of the plurality of the products in the first shelf image.

3. The information processing device according to claim 1 , wherein the sensor is installed above the shelf.

4. The information processing device according to claim 1 , wherein the processor is further configured to execute the instructions to:
generate the shelf image including a highlighting of the first product and the second product; and
display the shelf image on the display.

5. The information processing device according to claim 1 , wherein the processor is further configured to execute the instructions to:
count a number of times the customer's hand moved toward the second product based on the time series of the images;
generate the shelf image including the image of the plurality of the products displayed on the shelf and an index value based on the number of times, the index value being displayed at a position of the image of the second product; and
display the shelf image on the display.

6. The information processing device according to claim 5 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the second product and selected from among a plurality of colors,
wherein the colors are preset to a heat map and the predetermined color is changeable according to the number of times.

7. The information processing device according to claim 6 , wherein the image includes a legend indicating an association between the number of times and the plurality of the colors.

8. The information processing device according to claim 3 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the second product,
wherein an intensity of the predetermined color is changeable according to the number of times.

9. An information processing method, by at least one processor, comprising:
receiving an operation for designating a first product by a user among a plurality of products displayed on a shelf;
determining, based on time series of images captured the shelf, a second product touched by a customer who touched the first product;
receiving sensing data representing movement of a customer's hand toward the shelf;
determining, based on the time series of the images and the sensing data, the second product touched by the customer who touched the first product;
measuring a distance between the customer's hand and a sensor to determine the second product touched by the customer who touched the first product based on the distance; and
displaying, on a display, a shelf image including an image of the plurality of products,
wherein the shelf image includes the first product and the second product.

10. The information processing method according to claim 9 , wherein the shelf image is a second shelf image,
wherein the information processing method further comprises:
displaying, on the display, a first shelf image including an image of the plurality of the products; and
receiving, the operation for designating the first product by the user with the image of the plurality of the products in the first shelf image.

11. The information processing method according to claim 9 , wherein the sensor is installed above the shelf.

12. The information processing method according to claim 9 , further comprising:
generating the shelf image including a highlighting of the first product and the second product; and
displaying the shelf image on the display.

13. The information processing method according to claim 9 , further comprising:
counting a number of times the customer's hand moved toward the second product based on the time series of the images;
generating the shelf image including the image of the plurality of the products displayed on the shelf and an index value based on the number of times, the index value being displayed at a position of the image of the second product; and
displaying the shelf image on the display.

14. The information processing method according to claim 13 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the second product and selected from among a plurality of colors,
wherein the colors are preset to a heat map and the predetermined color is changeable according to the number of times.

15. The information processing method according to claim 11 , wherein the image indicates the index value with a predetermined color which is superimposed on the position of the image of the second product,
wherein an intensity of the predetermined color is changeable according to the number of times.

16. A non-transitory computer-readable storage medium having a program causing a computer to:
receive an operation for designating a first product by a user among a plurality of products displayed on a shelf;
determine, based on time series of images captured the shelf, a second product touched by a customer who touched the first product;
receive sensing data representing movement of a customer's hand toward the shelf;
determine, based on the time series of the images and the sensing data, the second product touched by the customer who touched the first product;
measure a distance between the customer's hand and a sensor to determine the second product touched by the customer who touched the first product based on the distance; and
display, on a display, a shelf image including an image of the plurality of products,
wherein the shelf image includes the first product and the second product.

Assignees

• Nec Corp

Inventors

• Kobayashi Jun
• Saito Shigetsu

Classifications

• G09G3/20

  for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix {no fixed position being assigned to or needed to be assigned to the individual characters or partial characters} · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06Q30/02 Primary

  Marketing; Price estimation or determination; Fundraising · CPC title

• G06Q30/06

  Buying, selling or leasing transactions · CPC title