Systems and methods of mapping an interior space of a product storage facility

Patent metadata

Field	Value
Publication number	US-12380400-B2
Application number	US-202217966580-A
Country	US
Kind code	B2
Filing date	Oct 14, 2022
Priority date	Oct 14, 2022
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

Systems and methods for use in mapping an interior space of a product storage facility include at least one sensor that captures distance measurement data with respect to an interior space of the product storage facility. A computing device obtains a first image representing a 2-dimensional map of the interior space of the product storage facility and processes this image to define a boundary of the interior space of the product storage facility and detect individual structures located within the interior space of the product storage facility. Then, the computing device defines separate department areas, assigns a department label to each of the separate department areas, and converts the 2-dimensional map representing the detected structures and the defined separate department areas and the department labels assigned to the separate department areas into a second image representing a 3-dimensional map of the interior space of the product storage facility.

First claim

What is claimed is:

1. A system for use in mapping an interior space of a product storage facility, the system comprising:
at least one sensor configured to capture distance measurement data with respect to at least one portion of the interior space of the product storage facility; and
a computing device including a control circuit, the computing device being communicatively coupled to the at least one sensor, the control circuit being configured to:
obtain a first image representing a 2-dimensional map of the interior space of the product storage facility, the first image being based on the distance measurement data; and
process the first image to:
define a boundary of the interior space of the product storage facility;
detect individual ones of structures located within the interior space of the product storage facility;
based on detection of groupings of multiple of the individual ones of the structures, detect separate department areas of the interior space of the product storage facility and a size of each of the separate department areas;
based on the size of each of the separate department areas, assign a department label to each of the separate department areas; and
convert the first image including the 2-dimensional map representing the individual ones of the structures and the separate department areas and department labels assigned to respective ones of the separate department areas into a second image representing a 3-dimensional map of the interior space of the product storage facility.

2. The system of claim 1 , further comprising a motorized robotic unit that includes the at least one sensor and that is configured to move about the interior space of the product storage facility to capture the distance measurement data.

3. The system of claim 1 , wherein the at least one sensor comprises a light detection and ranging (LIDAR) sensor.

4. The system of claim 1 , wherein the control circuit is further configured to generate at least one of:
a first set of virtual boundary lines, wherein each virtual boundary line of the first set of virtual boundary lines surrounds one of the separate department areas of the interior space of the product storage facility; and
a second set of virtual boundary lines that surround a defined boundary of the interior space of the product storage facility.

5. The system of claim 4 ,
further comprising an electronic database configured to store data indicating known dimensions and layout of the structures located within the interior space of the product storage facility and indicating known identities of the separate department areas of the interior space of the product storage facility; and
wherein the control circuit is further configured to correlate each virtual boundary line of the first set of virtual boundary lines to the known dimensions and layout of the structures located within the interior space of the product storage facility and to the known identities of the separate department areas to define the separate department areas of the interior space of the product storage facility and to assign each department label to its respective separate department area of the interior space of the product storage facility.

6. The system of claim 5 , wherein the control circuit is further configured to obtain the known dimensions and layout of the structures located within the interior space of the product storage facility and, based on the known dimensions and layout of the structures located within the interior space of the product storage facility, to assign a height to each of the structures in the 3-dimensional map of the interior space of the product storage facility.

7. The system of claim 1 , wherein the control circuit is further configured to assign a different color on the 3-dimensional map to the structures located in each of the separate department areas defined within the interior space of the product storage facility, wherein each of the structures located within a separate department area are assigned an identical color.

8. The system of claim 1 , wherein the control circuit is further configured to process the first image via at least one of splining, linear regression, and nearest neighbor techniques to define the boundary of the interior space of the product storage facility and to detect the individual ones of the structures located within the interior space of the product storage facility.

9. The system of claim 1 , wherein the control circuit is further configured to process the second image by adding depth information to the 3-dimensional map of the interior space of the product storage facility to:
detect individual ones of product storage bins located on the structures detected within the interior space of the product storage facility; and
define partitions between adjacent ones of the individual ones of product storage bins.

10. The system of claim 9 , wherein the control circuit is further configured to add the depth information via a deep neural network trained at least by images of the structures located in the interior space of the product storage facility, images of the product storage bins located on the structures located in the interior space of the product storage facility, and images of products stored on the structures located in the interior space of the product storage facility.

11. A method of mapping an interior space of a product storage facility, the method comprising:
capturing, via at least one sensor, distance measurement data with respect to at least one portion of the interior space of the product storage facility;
obtaining, via a computing device including a control circuit and communicatively coupled to the at least one sensor, the distance measurement data captured by the at least one sensor;
obtaining, via the computing device, a first image representing a 2-dimensional map of the interior space of the product storage facility, the first image being based on the distance measurement data; and
processing, via the control circuit of the computing device, the first image to:
define a boundary of the interior space of the product storage facility;
detect individual ones of structures located within the interior space of the product storage facility;
based on detection of groupings of multiple of the individual ones of the structures, detect separate department areas of the interior space of the product storage facility and a size of each of the separate department areas;
based on a the size of each of the separate department areas, assign a department label to each of the separate department areas; and
convert the first image including the 2-dimensional map representing the individual ones of the structures and the separate department areas and department labels assigned to respective ones of the separate department areas into a second image representing a 3-dimensional map of the interior space of the product storage facility.

12. The method of claim 11 , further comprising capturing the distance measurement data via a motorized robotic unit that includes the at least one sensor and that is configured to move about the interior space of the product storage facility to capture the distance measurement data.

13. The method of claim 11 , wherein the at least one sensor is a light detection and ranging (LIDAR) sensor.

14. The method of claim 11 , further comprising generating, via the control circuit, at least one of:
a first set of virtual boundary lines, wherein each virtual boundary line of the first set of virtual boundary lines surrounds one of the separate department areas of the interior space of the product storage facility;

Assignees

• Walmart Apollo Llc

Inventors

• Puviyarasu Paarvendhan
• Chen Yilun
• Polusani Harish
• Cantor Adam
• Ellison Benjamin R

Classifications

• G06V20/36

  Indoor scenes · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V2201/07

  Target detection · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title