Investigation system for finding lost objects

Patent metadata

Field	Value
Publication number	US-11544924-B1
Application number	US-202016842900-A
Country	US
Kind code	B1
Filing date	Apr 8, 2020
Priority date	Apr 8, 2019
Publication date	Jan 3, 2023
Grant date	Jan 3, 2023

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for finding lost objects. In some implementations, a request for a location of an item is obtained. Current video data from one or more cameras is obtained. It is determined that the item is not shown in the current video data. Sensor data corresponding to historical video data is obtained. Events that likely occurred with the item and corresponding likelihoods for each of the events are determined. A likely location for the item is determined based on the likelihoods determined for the events. An indication of the likely location of the item is provided.

First claim

What is claimed is:

1. A method comprising:
obtaining a request for a location of an item;
obtaining current video data from one or more cameras;
determining that the item is not shown in the current video data;
obtaining sensor data corresponding to historical video data;
determining events that likely occurred with the item and corresponding likelihoods for each of the events;
determining a likely location for the item based on the likelihoods determined for the events; and
providing an indication of the likely location of the item.

2. The method of claim 1 , comprising:
determining one or more additional likely locations for the item based on the likelihoods determined for the events; and
providing one or more indications of each of the one or more additional likely locations of the item.

3. The method of claim 2 , wherein determining a likely location for the item and determining one or more additional likely locations comprises:
obtaining a location count threshold;
based on the determined events that likely occurred, determining possible locations for the item; and
based on the likelihoods determined for the events, selecting from the possible locations the most likely locations, wherein a number of locations selected is equal to or less than the location count threshold.

4. The method of claim 1 , wherein determining a likely location comprises:
obtaining a location likelihood threshold;
based on the determined events that likely occurred, determining one or more possible locations for the item;
based on the likelihoods determined for the events, assigning a likelihood for each of the one or more possible locations;
comparing the likelihoods for each of the one or more possible locations with the location likelihood threshold; and
selecting a possible location with an assigned likelihood greater than or equal to the location likelihood threshold.

5. The method of claim 1 , wherein the sensor data comprises stored video data.

6. The method of claim 1 , wherein the sensor data comprises metadata extracted from the historical video data.

7. The method of claim 1 , wherein obtaining the sensor data comprises:
obtaining a time-period; and
accessing historical video data that was created within the obtained time-period.

8. The method of claim 1 , wherein determining that the item is not shown in the current video data comprises:
obtaining an appearance model of the item;
identifying objects within the current video data; and
comparing the identified objects with the appearance model of the item.

9. The method of claim 1 , wherein determining that the item is not shown in the current video data comprises:
obtaining a confidence threshold;
analyzing the current video data;
based on the analysis, determining a confidence of whether the item is in the current video data; and
determining that the confidence is below the confidence threshold.

10. The method of claim 9 , wherein determining a confidence comprises providing the analyzed video data to an inference model leveraging a machine-learning network.

11. The method of claim 1 , wherein determining events that likely occurred with the item and corresponding likelihoods for each of the events comprises:
extracting data points from the obtained sensor data;
based on the extracted data points, identifying at least one of the item, an owner of the item, a person other than the owner, an action of the owner, or an action of a person other than the owner; and
determining a likelihood that the one or more identifications are correct.

12. The method of claim 1 , wherein determining events that likely occurred with the item and corresponding likelihoods for each of the events comprises:
extracting data points from the obtained sensor data;
based on the extracted data points, identifying a pattern associated with the item;
based on the identified pattern, determining events that likely occurred with the item; and
based on the identified patterned, determining likelihoods for each of the determined events.

13. The method of claim 1 , wherein determining events that likely occurred with the item and corresponding likelihoods for each of the events comprises:
obtaining a likelihood threshold;
extracting data points from the obtained sensor data;
based on the extracted data points, determining one or more possible events that may have occurred with the item;
determining a likelihood for each of the one or more possible events;
comparing the one or more likelihoods for each of the one or more possible events with the likelihood threshold; and
selecting one or more events of the one or more possible events that have a likelihood at or above the likelihood threshold.

14. The method of claim 13 , wherein determining a likelihood comprises providing the extracted data points to an inference model leveraging a machine-learning network.

15. The method of claim 1 , wherein providing an indication of the likely location of the item comprises at least one of providing a textual description of the likely location, an image of the likely location, a clip of the likely location, or a floor plan having the likely location marked thereon.

16. The method of claim 1 , comprising:
receiving input to correct an event of the events that likely occurred with the item;
determining a new likelihood for each of the previously determined events; and
providing a new indication of the likely location of the item.

17. The method of claim 16 , wherein receiving input to correct an event comprises:
receiving an indication of a selection of the event; and
receiving an instruction to either verify the event or deny the event.

18. The method of claim 17 , wherein determining a new likelihood for each of the previously determined events comprises increasing the likelihood of the event if the instruction is to verify the event or decreasing the likelihood of the event if the instruction is to deny the event.

19. A system comprising:
one or more sensors, including one or more cameras; and
a computer having one or more processors, the computer configured to:
obtain a request for a location of an item;
obtain current video data from the one or more cameras;
determine that the item is not shown in the current video data;
obtain sensor data corresponding to historical video data;
determine events that likely occurred with the item and corresponding likelihoods for each of the events;
determine a likely location for the item based on the likelihoods determined for the events; and
provide an indication of the likely location of the item.

20. One or more non-transitory computer-readable media storing a computer program, the program comprising instructions that when executed by one or more processing devices cause the one or more processing devices to perform operations comprising:
obtaining, by the one or more processing devices, a request for a location of an item;
obtaining, by the one or more processing devices, current video data from one or more cameras;
determining, by the one or more processing devices, that the item is not shown in the current video data;
obtaining, by the one or more processing devices, sensor data corresponding to historical video data;
determining, by the one or more processing devices, events that likely occurred with the item and corresponding likelihoods for each of the events;
determining, by the one or more processing devices, a likely location for the item based on the likelihoods determine

Assignees

• Alarm Com Inc

Inventors

• Lee Jangwon
• Roberts Aaron Lee
• Jutras Bret
• Beach Allison
• Madden Donald Gerard

Classifications

• G06V20/44

  Event detection · CPC title

• G08B21/182

  Level alarms, e.g. alarms responsive to variables exceeding a threshold · CPC title

• G08B31/00

  Predictive alarm systems characterised by extrapolation or other computation using updated historic data · CPC title

• G06V20/40 Primary

  in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44) · CPC title

• G06V2201/10

  Recognition assisted with metadata · CPC title