Virtualizing objects using object models and object position data

Patent metadata

Field	Value
Publication number	US-12260640-B2
Application number	US-202418596078-A
Country	US
Kind code	B2
Filing date	Mar 5, 2024
Priority date	Dec 13, 2017
Publication date	Mar 25, 2025
Grant date	Mar 25, 2025

Abstract

Described herein are techniques for receiving, from a first device, first data about a physical object located in a physical space. The techniques further including storing, in a catalog of objects generated for the physical space, information about the physical object and indicating at least a first location of the physical object in the physical space. The techniques include receiving, from a second device, second data about the physical space, the second data sent from the second device upon or after an occurrence of an event. The techniques include determining, based on the second data, at least an impact area of the event to the physical space and determining, based on the catalog of objects and the second data, whether the physical object is impacted by the event. The techniques include causing an output indicating whether the physical object is impacted by the event.

First claim

What is claimed is:

1. A method comprising:
receiving, from a first device, first data about a physical object located in a physical space;
storing, in a catalog of objects generated for the physical space, information about the physical object and indicating at least a first location of the physical object in the physical space;
receiving, from a second device, second data about the physical space, the second data sent from the second device upon or after an occurrence of an event, the second device being the same as or different from the first device;
determining, based on the second data, at least an impact area of the event to the physical space;
determining, based on the catalog of objects and the second data, whether the physical object is impacted by the event; and
causing an output indicating whether the physical object is impacted by the event.

2. The method of claim 1 , wherein receiving the first data includes receiving sensor data generated by one or more sensors of the first device.

3. The method of claim 1 , further comprising generating the information by at least:
determining, based on the first data, the first location of the physical object;
mapping the physical object to one or more corresponding physical object models within an object model database;
identifying that the physical object is to be associated with the first location; and
including, in the information, an indication of the physical object model and the first location of the physical object.

4. The method of claim 1 , wherein storing the information in the catalog of objects comprises storing a representation of the physical object based on the first data.

5. The method of claim 4 , wherein the first data includes image data, and wherein the representation is generated based on the image data.

6. The method of claim 1 , wherein the first data includes depth information associated with the physical object.

7. The method of claim 6 , wherein the information about the physical object includes a first position of the physical object determined using the depth information associated with the physical object.

8. The method of claim 7 , wherein the first location comprises the first position and a first orientation.

9. The method of claim 1 , wherein the second data about the physical space comprises image data.

10. The method of claim 9 , wherein the impact area is determined based on user input indicating a boundary of the impact area in at least one image.

11. The method of claim 9 , wherein the image data comprises at least one of color data or texture data corresponding to the physical space or the physical object and the impact area is determined by using at least one of the color data or the texture data.

12. A system comprising:
one or more processors; and
one or more memory storing instructions that upon execution by the one or more processors, configure the system to:
receive, from a first device, first data about a physical object located in a physical space;
store, in a catalog of objects generated for the physical space, information about the physical object and indicating at least a first location of the physical object in the physical space;
receive, from a second device, second data about the physical space, the second data sent from the second device upon or after an occurrence of an event, the second device being the same as or different from the first device;
determine, based on the second data, at least an impact area of the event to the physical space;
determine, based on the catalog of objects and the second data, whether the physical object is impacted by the event; and
cause an output indicating whether the physical object is impacted by the event.

13. The system of claim 12 , wherein the second data corresponds to a floorplan layout of the physical space.

14. The system of claim 13 , wherein the instructions, upon execution by the one or more processors, further configure the system to determine the impact area based on user input indicating a boundary of the impact area and on the floorplan layout.

15. The system of claim 12 , wherein the instructions, upon execution by the one or more processors, further configure the system to associate the first location with the impact area if the first location is determined to be within the impact area.

16. The system of claim 12 , wherein the instructions, upon execution by the one or more processors, further configure the system to determine that the physical object is impacted by the event if the first location is determined to be within the impact area.

17. The system of claim 12 , wherein the instructions, upon execution by the one or more processors, further configure the system to:
determine, based on the second data, a second location of the physical object, wherein the physical object is determined to be impacted by the event if a determination is made that the first location does not correspond to the second location of the physical object.

18. One or more non-transitory computer-readable storage media storing instructions that upon execution on a system, cause the system to perform operations comprising:
receiving, from a first device, first data about a physical object located in a physical space;
storing, in a catalog of objects generated for the physical space, information about the physical object and indicating at least a first location of the physical object in the physical space;
receiving, from a second device, second data about the physical space, the second data sent from the second device upon or after an occurrence of an event, the second device being the same as or different from the first device;
determining, based on the second data, at least an impact area of the event to the physical space;
determining, based on the catalog of objects and the second data, whether the physical object is impacted by the event; and
causing an output indicating whether the physical object is impacted by the event.

19. The one or more non-transitory computer-readable storage media of claim 18 , wherein the operations further comprise:
determining, based on the second data, a second location of the physical object, wherein the second data about the physical space includes image data, wherein the image data comprises at least one of color data or texture data corresponding to the physical object, and wherein the physical object is determined to be impacted by the event if a determination is made that at least one of the color data or the texture data does not correspond to a color property of the physical object or a texture property of the physical object stored in the catalog.

20. The one or more non-transitory computer-readable storage media of claim 18 , wherein the operations further comprise:
determining, based on the second data, a second location of the physical object, wherein causing the output includes identifying the physical object and an impact assessment for the physical object on a user interface of the second device.

Assignees

• Lowes Companies Inc

Inventors

• Sheffield Mason E
• Shabtai Josh

Classifications

• G06F16/9537

  Spatial or temporal dependent retrieval, e.g. spatiotemporal queries · CPC title

• G06F16/5854

  using shape and object relationship · CPC title

• G06N20/00

  Machine learning · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title