Store shelf imaging system and method

Patent metadata

Field	Value
Publication number	US-10210603-B2
Application number	US-201615294937-A
Country	US
Kind code	B2
Filing date	Oct 17, 2016
Priority date	Oct 17, 2016
Publication date	Feb 19, 2019
Grant date	Feb 19, 2019

Abstract

A store profile generation system includes a mobile base and an image capture assembly mounted on the base. The assembly includes at least one image capture device for acquiring images of product display units in a retail environment. A control unit acquires the images captured by the at least one image capture device at a sequence of locations of the mobile base in the retail environment. The control unit extracts product-related data from the acquired images and generates a store profile indicating locations of products and their associated tags throughout the retail environment, based on the extracted product-related data. The store profile can be used for generating new product labels for a sale in an appropriate order for a person to match to the appropriate locations in a single pass through the store.

First claim

What is claimed is:

1. A continuous motion store profile generation system comprising:
a robotic mobile base including an operatively associated primary navigation component configured to determine a location of the robotic mobile base within a retail environment, the retail environment including a plurality of product display units arranged in a plurality of aisles, each product display unit including a plurality of product display fixtures, a plurality of products arranged on one or more of the product display fixtures, and a plurality of tags attached to the product display fixtures, each tag including product-related data including a barcode associated with a respective product proximately located on the product display fixture near the tag;
a digital image capture assembly mounted on the robotic mobile base, the digital image capture assembly including a plurality of digital image capture devices for acquiring digital images of the product display units;
a master control unit operatively associated with the digital image capture assembly including an operatively associated memory which stores instructions and a processor for executing the instructions, the control unit instructions including instructions for processing the digital images acquired by the plurality of digital image capture devices at a sequence of locations during a continuous movement of the robotic mobile base in the retail environment, thereafter extracting the product-related data from the digital images of the product display units captured by the plurality of image digital capture devices during the continuous movement of the robotic mobile base, and generating a store profile indicating locations of the products and/or tags throughout the retail environment based on the extracted product-related data, a spatial characterization of the digital image capture assembly, and the locations of the robotic mobile base at the sequence of locations provided by the primary navigation component during continuous movement of the robotic mobile base during an interval of time that the digital images were acquired and an image-based accuracy correction component configured to generate a corrected location of the robotic mobile base by determining a shift distance associated with aligning overlapping portions of sequential overlapping digital images, the shift distance applied to the location of the robotic mobile base provided by the primary navigation component.

2. The continuous motion store profile generation system according to claim 1 , wherein the image-based accuracy correction performs one or more of a cross correlation function between the acquired sequentially overlapping digital images to calculate the shift distance, and a Harris Corners Detector function to identify a most frequent shift distance associated with the acquired sequentially overlapping digital images to calculate the shift distance.

3. The continuous motion store profile generation system according to claim 1 , wherein the primary navigation component is configured to determine the location of the robotic mobile base with a positional error substantially equal to or greater than +/− 40 mm and the image-based accuracy correction process is configured to generate the corrected location of the robotic mobile base with a positional error less than +/− 40 mm.

4. The continuous motion store profile generation system according to claim 1 , wherein the image-based accuracy correction performs a cross correlation function configured to:
overlap the sequential overlapping digital images by a horizontal separation distance traveled by the robotic mobile base from a first imaging location associated with acquiring a first of the sequential overlapping digital images to a second imaging location associated with acquiring a second of the sequential overlapping digital images, the horizontal separation distance calculated based on the location of the robotic mobile base along the first aisle provided by the primary navigation system;
determine an optimal shift that yields a minimum sum of absolute differences associated with the overlapped sequential overlapping digital images to horizontally align the overlapping portions of the sequential overlapping digital images; and
apply the optimal shift as a correction to the location of the robotic mobile base along the aisle at the second imaging location provided by the primary navigation component.

5. The continuous motion store profile generation system according to claim 4 , wherein the cross correlation function is performed for a plurality of digital images captured at one or both of the first imaging location and the second imaging location, and the correction to the location of the robotic mobile base along the aisle is based on an average or median of the optimal shifts associated with the plurality of digital images captured at one or both of the first imaging location and the second imaging location.

6. The continuous motion store profile generation system according to claim 4 , wherein a search range of S opt associated with finding the optimal shift that yields the minimum sum of absolute differences is limited based on a priori information associated with the robotic mobile base and/or the retail environment.

7. The continuous motion store profile generation system according to claim 1 , further comprising:
a configuration component configured to spatially characterize the plurality of digital image capture devices to map pixels of a captured digital image to locations in a real space.

8. A store profile generation system comprising:
a robotic mobile base including an operatively associated primary navigation component configured to determine a location of the robotic mobile base within a retail environment, the retail environment including a plurality of product display units arranged in a plurality of aisles, each product display unit including a plurality of product display fixtures, a plurality of products arranged on one or more of the product display fixtures, and a plurality of tags attached to the product display fixtures, each tag including product-related data including a barcode associated with a respective product proximately located on the product display fixture near the tag;
a high resolution (HR) digital image capture assembly mounted on the robotic mobile base, the digital image capture assembly including a plurality of high resolution (HR) digital image capture devices configured at a first resolution and a first field of view (FOV) to acquire HR digital images of the product display units; and
a master control unit operatively associated with the HR digital image capture assembly including an operatively associated memory which stores instructions and a processor for executing the instructions, the control unit instructions including instructions for processing the HR digital images acquired by the plurality of HR digital image capture devices at a sequence of locations during a continuous movement of the robotic mobile base in the retail environment, thereafter extracting the product-related data from the HR digital images of the product display units captured by the plurality of HR digital image capture devices during the continuous movement of the robotic mobile base, and generating a store profile indicating locations of the products and/or tags throughout the retail environment based on the extracted product-related data, a spatial characterization of the image capture assembly, and the locations of the robotic mobile base at the sequence of locations provided by the primary navigation component during continuous movement of the robotic mobile base during an interval of time that the HR digital images were acquired and an image-based accuracy correction component

Assignees

• Conduent Business Services Llc

Inventors

• Venable Dennis L
• Wu Wencheng
• Wade Thomas F
• Shen Ethan
• Rizzolo Charles D

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• H04N5/2253

  Electricity · mapped topic

• G06T7/0004

  Industrial image inspection · CPC title