Shelf space allocation management device and shelf space allocation management method

Patent metadata

Field	Value
Publication number	US-10872264-B2
Application number	US-201816125383-A
Country	US
Kind code	B2
Filing date	Sep 7, 2018
Priority date	Jan 22, 2015
Publication date	Dec 22, 2020
Grant date	Dec 22, 2020

Abstract

A shelf space allocation management device manages products allocated on shelves aligned in a store by use of an image captured by an imaging device. The shelf space allocation management device acquires an image including a position assumed to be changed in allocation status of each product on each shelf, it determines whether the type and the allocation status of each product reflected in the image match the predetermined type and the predetermined allocation status; then, it determines whether to execute a product allocation inspection based on the determination result. Herein, the shelf space allocation management device specifies a position at which a person conducts a behavior to cause any change in the allocation status of each product on each shelf, and therefore it may control the imaging device to capture an image including the position. It is possible to carry out a product allocation inspection for each period determined in advance depending on the type of each product, or it is possible to carry out a product allocation inspection being triggered by a customer purchasing each product.

First claim

The invention claimed is:

1. A shelf space allocation management device for managing a plurality of products allocated on a shelf among a plurality of shelves using a mobile imaging device movable in a store, the shelf space allocation management device comprising:
a memory storing instructions; and
a processor coupled to the memory and configured to execute the instructions to:
acquire a first image taken by the mobile imaging device configured to move in a store under an automatic tracking mode configured to track a clerk;
determine whether the clerk located in the store and captured in the first image has performed a predetermined action on the shelf;
specify position information of the shelf for a product allocation inspection, wherein the position information is assumed to be changed in product allocation due to the predetermined action on the shelf;
change the automatic tracking mode to a position control mode directing movement according to position information of the mobile imaging device and a target position determined in advance after specifying the position information of the shelf;
move a drone having the mobile imaging device and configured to automatically track a person located in the store to an image-capture position in a direction calculated according to the position information of the shelf in the position control mode;
adjust the direction of the mobile imaging device upon determining that the mobile imaging device arrives at the image-capture position in the position control mode;
control the mobile imaging device to capture a second image on the shelf at the image-capture position;
determine whether a type and an allocation status of a plurality of products reflected in the second image match a predetermined type and a predetermined allocation status of the plurality of products; and
execute the product allocation inspection based on a determination result of the type and the allocation status of the product.

2. The shelf space allocation management device according to claim 1 , wherein
the predetermined action is determined using a third image of the person captured by a secondary imaging device fixed at a predetermined position in the store as to whether the person has displayed the product on the shelf.

3. The shelf space allocation management device according to claim 1 , wherein
the predetermined action is determined using a third image of the person captured by a secondary imaging device fixed at a predetermined position in the store as to whether the person has repeatedly extended an arm toward at least one of the plurality of products on the shelf a predetermined number of times in a predetermined period of time.

4. The shelf space allocation management device according to claim 1 , wherein the processor is further configured to execute the product allocation inspection in a first time interval which is determined for each product among the plurality of products in advance or in a second time interval which is determined for each shelf among the plurality of shelves in advance.

5. The shelf space allocation management device according to claim 1 , wherein the processor is further configured to estimate an allocated orientation of each product, wherein the allocated orientation comprises at least one of a vertical or horizontal orientation of each product or a backward or forward orientation of each product.

6. The shelf space allocation management device according to claim 1 , wherein the processor is further configured to determine that the allocation status of the plurality of products matches the predetermined allocation status of the plurality of products when a number of products reflected in the second image is greater than or equal to a predetermined number of products.

7. A shelf space allocation management method for managing a plurality of products allocated on a shelf among a plurality of shelves using a mobile imaging device movable in a store, the shelf space allocation management method comprising:
acquiring a first image taken by the mobile imaging device configured to move in a store under an automatic tracking mode configured to track a clerk;
determining whether the clerk located in the store and captured in the first image has performed a predetermined action on the shelf;
specifying position information of the shelf for a product allocation inspection, wherein the position information is assumed to be changed in product allocation due to the predetermined action on the shelf;
changing the automatic tracking mode to a position control mode directing movement according to position information of the mobile imaging device and a target position determined in advance after specifying the position information of the shelf;
moving a drone having the mobile imaging device and configured to automatically track a person located in the store to an image-capture position in a direction calculated according to the position information of the shelf in the position control mode;
adjusting the direction of the mobile imaging device upon determining that the mobile imaging device arrives at the image-capture position in the position control mode;
controlling the mobile imaging device to capture a second image on the shelf at the image-capture position;
determining whether a type and an allocation status of a plurality of products reflected in the second image match a predetermined type and a predetermined allocation status of the plurality of products; and
executing the product allocation inspection based on a determination result of the type and the allocation status of the product.

8. The shelf space allocation management method according to claim 7 , wherein
the predetermined action is determined using a third image of the person captured by a secondary imaging device fixed at a predetermined position in the store as to whether the person has displayed the product on the shelf.

9. The shelf space allocation management method according to claim 7 , wherein
the predetermined action is determined using a third image of the person captured by a secondary imaging device fixed at a predetermined position in the store as to whether the person has repeatedly extended an arm toward at least one of the plurality of products on the shelf a predetermined number of times in a predetermined period of time.

10. The shelf space allocation management method according to claim 7 , further comprising executing the product allocation inspection in a first time interval which is determined for each product among the plurality of products in advance or in a second time interval which is determined for each shelf among the plurality of shelves in advance.

11. The shelf space allocation management method according to claim 7 , further comprising estimating an allocated orientation of each product, wherein the allocated orientation comprises at least one of a vertical or horizontal orientation of each product or a backward or forward orientation of each product.

12. The shelf space allocation management method according to claim 7 , further comprising determining that the allocation status of the plurality of products matches the predetermined allocation status of the plurality of products when a number of products reflected in the second image is greater than or equal to a predetermined number of products.

13. A non-transitory computer readable medium having stored therein a shelf space allocation management program causing a computer to perform a shelf space allocation management process for managing a plurality of products allocated on a shelf among a plurality of shelves using a mobile imaging device movable in a store,

Assignees

• Nec Corp

Inventors

• Yamashita Nobuyuki

Classifications

• H04N7/18 Primary

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title

• G06V10/75

  Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries · CPC title

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/17

  using hand-held instruments · CPC title