Method and system for moving object detection with single camera

Patent metadata

Field	Value
Publication number	US-10019637-B2
Application number	US-201615349556-A
Country	US
Kind code	B2
Filing date	Nov 11, 2016
Priority date	Nov 13, 2015
Publication date	Jul 10, 2018
Grant date	Jul 10, 2018

Abstract

Disclosed are systems and methods for detecting moving objects. A computer-implemented method for detecting moving objects comprises obtaining a streaming video captured by a camera; extracting an input image sequence including a series of images from the streaming video; tracking point features and maintaining a set of point trajectories for at least one of the series of images; measuring a likelihood for each point trajectory to determine whether it belongs to a moving object using constraints from multi-view geometry; and determining a conditional random field (CRF) on an entire frame to obtain a moving object segmentation.

First claim

What is claimed is:

1. A computer-implemented method for detecting moving objects, comprising:
obtaining a streaming video captured by a camera;
extracting an input image sequence including a series of images from the streaming video;
tracking point features and maintaining a set of point trajectories for at least one of the series of images;
measuring a likelihood for each point trajectory to determine whether it belongs to a moving object using constraints from multi-view geometry;
determining a conditional random field (CRF) on an entire frame to obtain a moving object segmentation,
wherein the method further comprises, for each frame T:
computing an optical flow and a point trajectory;
estimating fundamental matrices and a trifocal tensor;
computing an epipolar moving objectness score and a trifocal moving objectness score for each trajectory; and
forming the CRF on superpixels to determine moving labels.

2. The method of claim 1 , wherein the camera comprises a monocular camera.

3. The method of claim 1 , wherein the constraints from multi-view geometry comprise at least one of: an epipolar constraint between two-view and trifocal constraints from three-view.

4. The method of claim 3 , wherein the epipolar constraint is calculated based at least in part on an epipolar moving objectness score for a pair of point correspondence as follows:
ϕ( x i τ ,x i τ′ )= d pl ( F τ τ′ x i τ′ ,x i τ )+ d pl ( x i τ′ ,x i τ F τ τ′ ),

where F τ τ′ x i τ′ and x i τ F τ τ′ define relative epipolar lines in each view and a function d pl (·) computes a point to line distance.

5. The method of claim 4 , wherein the pair of point correspondence is determined based on an optical flow between consecutive frames.

6. The method of claim 4 , further comprising determining a weighted average of all epipolar moving objectness scores of a trajectory as follows:

Φ
⁡

(

z

i
,
τ

t

)

=

1

B
⁡

(

z

i
,
τ

t

)

⁢

∑

m
=

τ
+
1

t

⁢

β

t
-
m

⁢

∑

n
=
τ

m
-
1

⁢

ϕ
⁡

(

x
i
m

,

x
i
n

)

,

where β is a decay factor.

7. The method of claim 4 , further comprising determining another epipolar moving objectness score as follows:

Ψ
⁡

(

z

i
,
τ

t

)

=

1

ρ
⁡

(

z

i
,
τ

t

)

⁢

∑

m
=

τ
+
1

t

⁢

∑

n
=
τ

m
-
1

⁢

[

ϕ
⁡

(

x
i
m

,

x
i
n

)

>

Assignees

• Honda Motor Co Ltd

Inventors

• Chen Sheng
• Ayvaci Alper

Classifications

• G06V20/54 Primary

  of traffic, e.g. cars on the road, trains or boats · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06F18/2113

  by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06K2209/21

  Physics · mapped topic