Ground plane filtering of video events

Patent metadata

Field	Value
Publication number	US-12591980-B2
Application number	US-202318242147-A
Country	US
Kind code	B2
Filing date	Sep 5, 2023
Priority date	Mar 19, 2020
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

Methods, systems, and apparatus for ground plane filtering of video events are disclosed. A method includes obtaining a first set of images of a scene from a camera; determining a ground plane from the first set of images of the scene; obtaining a second set of images of the scene after the first set of images of the scene is obtained; determining that movement shown by a group of pixels in the second set of images of the scene satisfies motion criteria; determining that the ground plane corresponds with at least a portion of the group of pixels; and in response to determining that movement shown by the group of pixels in the second set of images of the scene satisfies motion criteria, and that the ground plane corresponds with at least a portion of the group of pixels, classifying the group of pixels as showing ground plane based motion.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
identifying a plurality of moving objects depicted in a set of images captured by a camera, the identifying comprising:
performing optical flow analysis on the set of images; and
based on the optical flow analysis, identifying a plurality of pixel groups each having an optical flow magnitude that satisfies a threshold optical flow magnitude,
wherein each moving object of the plurality of moving objects is represented by a corresponding pixel group of the plurality of pixel groups;
identifying, from the plurality of moving objects:
a first subset of moving objects that are each represented by a pixel group that overlaps at least in part with a depiction of a surface in the set of images; and
a second subset of moving objects that are each represented by a pixel group that does not overlap with the depiction of the surface in the set of images;
tracking, in subsequent images captured by the camera, motion of the first subset of moving objects; and determining not to track, in the subsequent images captured by the camera, motion of second subset of moving objects.

2 . The computer-implemented method of claim 1 , wherein identifying the plurality of moving objects further comprises:
determining that the plurality of pixel groups each has a uniformity of motion direction that satisfies motion criteria.

3 . The computer-implemented method of claim 1 , wherein identifying the first subset of moving objects comprises:
determining, for each moving object of the plurality of moving objects, a percentage of the corresponding pixel group representing the moving object that is in a ground plane of the set of images; and selecting, for inclusion in the first subset of moving objects, moving objects for which the percentage of the corresponding pixel group representing the moving object that is in the ground plane of the set of images satisfies a threshold percentage.

4 . The computer-implemented method of claim 1 , wherein identifying the first subset of moving objects comprises:
determining, for each moving object of the plurality of moving objects, a number of pixels of the corresponding pixel group representing the moving object that are in a ground plane of the set of images; and selecting, for inclusion in the first subset of moving objects, moving objects for which the number of pixels of the corresponding pixel group representing the moving object that are in the ground plane of the set of images satisfies a threshold number of pixels.

5 . The computer-implemented method of claim 1 , comprising:
in response to tracking the motion of the first subset of moving objects, determining that at least one ground plane motion event has occurred; and performing one or more actions based on the at least one ground plane motion event.

6 . The computer-implemented method of claim 1 , comprising:
obtaining a ground plane image including:
a first set of pixels corresponding to a ground plane region of a scene captured by the camera, each pixel of the first set of pixels having a first pixel value; and
a second set of pixels corresponding to a non-ground plane region of the scene captured by the camera, each pixel of the second set of pixels having a second pixel value,
wherein each moving object of the first subset of moving objects is represented by a pixel group that includes at least one pixel that has a pixel location corresponding to a pixel in the ground plane image that has the first pixel value.

7 . The computer-implemented method of claim 1 , wherein tracking the motion of the first subset of moving objects comprises:
generating a bounding box around each moving object of the first subset of moving objects; and tracking movements of the bounding boxes in the subsequent images captured by the camera.

8 . The computer-implemented method of claim 1 , wherein the surface depicted in the set of images comprises a walkable surface.

9 . The computer-implemented method of claim 1 , wherein the surface depicted in the set of images is defined in part by a horizon.

10 . The computer-implemented method of claim 1 , wherein the surface depicted in the set of images comprises a surface on which one or more objects appear to be positioned.

11 . The computer-implemented method of claim 1 , wherein the identifying the plurality of moving objects further comprises:
generating a bounding box around an object depicted in the set of images, wherein performing the optical flow analysis comprises determining an average optical flow magnitude for pixels within the bounding box.

12 . One or more non-transitory computer-readable media storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
identifying a plurality of moving objects depicted in a set of images captured by a camera, the identifying comprising:
performing optical flow analysis on the set of images; and
based on the optical flow analysis, identifying a plurality of pixel groups each having an optical flow magnitude that satisfies a threshold optical flow magnitude,
wherein each moving object of the plurality of moving objects is represented by a corresponding pixel group of the plurality of pixel groups;
identifying, from the plurality of moving objects:
a first subset of moving objects that are each represented by a pixel group that overlaps at least in part with a depiction of a surface in the set of images; and
a second subset of moving objects that are each represented by a pixel group that does not overlap with the depiction of the surface in the set of images;
tracking, in subsequent images captured by the camera, motion of the first subset of moving objects; and determining not to track, in the subsequent images captured by the camera, motion of second subset of moving objects.

13 . The one or more non-transitory computer-readable media of claim 12 , wherein identifying the plurality of moving objects further comprises:
determining that the plurality of pixel groups each has a uniformity of motion direction that satisfies motion criteria.

14 . The one or more non-transitory computer-readable media of claim 12 , wherein identifying the first subset of moving objects comprises:
determining, for each moving object of the plurality of moving objects, a percentage of the corresponding pixel group representing the moving object that is in a ground plane of the set of images; and selecting, for inclusion in the first subset of moving objects, moving objects for which the percentage of the corresponding pixel group representing the moving object that is in the ground plane of the set of images satisfies a threshold percentage.

15 . The one or more non-transitory computer-readable media of claim 12 , wherein the identifying the plurality of moving objects further comprises:
generating a bounding box around an object depicted in the set of images, wherein performing the optical flow analysis comprises determining an average optical flow magnitude for pixels within the bounding box.

16 . A system comprising:
one or more computers and one or more computer storage media storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
identifying a plurality of moving objects depicted in a set of images captured by a camera, the identifying comprising:
performing optical flow analysis on the set of images; and
based on th

Assignees

• Objectvideo Labs Llc

Inventors

• Ramanathan Narayanan
• Park Kyoung-Jin
• Qian Gang
• Beach Allison
• Madden Donald Gerard

Classifications

• G06V10/62

  relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title