Camera detection of object movement with co-occurrence

Patent metadata

Field	Value
Publication number	US-12579871-B2
Application number	US-202318108300-A
Country	US
Kind code	B2
Filing date	Feb 10, 2023
Priority date	Mar 24, 2020
Publication date	Mar 17, 2026
Grant date	Mar 17, 2026

Abstract

Methods, systems, and apparatus for camera detection of human activity with co-occurrence are disclosed. A method includes detecting a person in an image captured by a camera; in response to detecting the person in the image, determining optical flow in portions of a first set of images; determining that particular portions of the first set of images satisfy optical flow criteria; in response to determining that the particular portions of the first set of images satisfy optical flow criteria, classifying the particular portions of the first set of images as indicative of human activity; receiving a second set of images captured by the camera after the first set of images; and determining that the second set of images likely shows human activity based on analyzing portions of the second set of images that correspond to the particular portions of the first set of images classified as indicative of human activity.

First claim

What is claimed is:

1 . A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
maintaining, in memory and for use with a plurality of sets of images including a first set of images and for each of a plurality of different predetermined movement types for a corresponding object type, optical flow grid criteria, the plurality of different predetermined movement types comprising a predetermined type of movement for a type of an object; determining optical flow in pixel groups of the first set of images (i) that are captured by a camera and (ii) at least one of which depicts an object within a threshold distance of a property; determining whether particular pixel groups of the first set of images satisfy the optical flow grid criteria that i) indicate flow motion characteristics of the predetermined type of movement for the type of the object and ii) include criteria for at least one of optical flow magnitude or optical flow direction of the pixel groups of the first set of images, each of the particular pixel groups having the same location in each image of the first set of images; in response to determining that the particular pixel groups of the first set of images satisfy the optical flow grid criteria, classifying the particular pixel groups of the first set of images as indicative of the predetermined type of movement for the type of the object; receiving a second set of images captured by the camera after the first set of images; analyzing second pixel groups of the second set of images that correspond to the locations of the particular pixel groups of the first set of images classified as indicative of the predetermined type of movement for the type of the object; determining that the second set of images likely shows the predetermined type of movement for the type of the object using a result of analyzing of the second pixel groups of the second set of images that correspond to the locations of the particular pixel groups of the first set of images classified as indicative of the predetermined type of movement; and performing one or more actions for the property in response to classifying the particular pixel groups of the first set of images as indicative of the predetermined type of movement for the type of the object and determining that the second set of images likely shows the predetermined type of movement for the type of the object.

2 . The system of claim 1 , wherein determining that the second set of images likely shows the predetermined type of movement for the type of the object using the result of analyzing of the second pixel groups of the second set of images that correspond to the locations of the particular pixel groups of the first set of images classified as indicative of the predetermined type of movement for the type of the object comprises detecting optical flow in the second pixel groups of the second set of images that correspond to the locations of the particular pixel groups of the first set of images that are classified as indicative of the predetermined type of movement for the type of the object.

3 . The system of claim 1 , wherein classifying the particular pixel groups of the first set of images as indicative of the predetermined type of movement for the type of the object comprises determining that the particular pixel groups of the first set of images depict movement of another object that moves in co-occurrence with movement of the object.

4 . The system of claim 3 , wherein determining that the second set of images likely shows the predetermined type of movement for the type of the object using the result of analyzing of the second pixel groups of the second set of images that correspond to the locations of the particular pixel groups of the first set of images classified as indicative of the predetermined type of movement for the type of the object comprises:
determining, in response to detecting motion of the other object that moves in co-occurrence with movement of the object, that the second set of images likely shows the predetermined type of movement for the type of the object.

5 . The system of claim 1 , wherein classifying the particular pixel groups of the first set of images as indicative of the predetermined type of movement for the type of the object comprises determining that the particular pixel groups of the first set of images correspond to an object trajectory through a scene captured by the camera.

6 . The system of claim 5 , wherein determining that the second set of images likely shows the predetermined type of movement for the type of the object using the result of analyzing of the second pixel groups of the second set of images that correspond to the locations of the particular pixel groups of the first set of images classified as indicative of the predetermined type of movement for the type of the object comprises detecting motion along the object trajectory through the scene captured by the camera.

7 . The system of claim 1 , wherein determining that the second set of images likely shows the predetermined type of movement for the type of the object using the result of analyzing of the second pixel groups of the second set of images that correspond to the locations of the particular pixel groups of the first set of images classified as indicative of the predetermined type of movement for the type of the object comprises:
determining that a matching percentage between portions of the second set of images that exhibit optical flow and the particular pixel groups of the first set of images satisfies a threshold matching percentage.

8 . The system of claim 1 , the operations comprising generating a bounding box around the object, wherein:
classifying the particular pixel groups of the first set of images as indicative of the predetermined type of movement for the type of the object comprises determining that an amount of overlap between the particular pixel groups of the first set of images and the bounding box does not satisfy a threshold overlap.

9 . The system of claim 1 , wherein the particular pixel groups of the first set of images comprise segments of a grid overlaid on each image of the first set of images, the operations comprising generating a gridded representation of the particular pixel groups of the first set of images that are classified as indicative of the predetermined type of movement for the type of the object.

10 . The system of claim 9 , wherein the gridded representation includes binary representations indicating whether each portion of the first set of images is indicative of the predetermined type of movement for the type of the object.

11 . The system of claim 9 , wherein the gridded representation includes gradient representations indicating a degree to which each portion of the first set of images is indicative of the predetermined type of movement for the type of the object.

12 . The system of claim 9 , wherein performing the one or more actions comprises updating at least a portion of the grid using data from at least one of the first set of images or the second set of images.

13 . The system of claim 9 , wherein performing the one or more actions comprises:
receiving input that indicates an inaccuracy in the performance of the one or more actions; and in response to receiving the input that indicates the inaccuracy in the performance of the one or more actions, updating at least a portion of the grid using data from at least one of the first set of images or the sec

Assignees

• Objectvideo Labs Llc

Inventors

• Ramanathan Narayanan
• Beach Allison
• Qian Gang
• Madden Donald Gerard

Classifications

• H04N23/681

  Motion detection · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G08B13/19626

  optical details, e.g. lenses, mirrors or multiple lenses · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• H04N23/65

  Control of camera operation in relation to power supply · CPC title