Continuous Occlusion Models for Road Scene Understanding

Patent metadata

Field	Value
Publication number	US-2016137206-A1
Application number	US-201514879264-A
Country	US
Kind code	A1
Filing date	Oct 9, 2015
Priority date	Nov 13, 2014
Publication date	May 19, 2016
Grant date	—

Abstract

Systems and methods are disclosed for road scene understanding of vehicles in traffic by capturing images of traffic with a camera coupled to a vehicle; generating a continuous model of occlusions with a continuous occlusion mode for traffic participants to enhance point track association accuracy without distinguishing between moving and static objects; applying the continuous occlusion model to handle visibility constraints in object tracks; and combining point track association and soft object track modeling to improve 3D localization accuracy.

First claim

What is claimed is:

1 . A method for road scene understanding of vehicles in traffic, comprising
capturing images of traffic with a camera coupled to a vehicle; generating a continuous model of occlusions with a continuous occlusion mode for traffic participants to enhance point track association accuracy without distinguishing between moving and static objects; applying the continuous occlusion model to handle visibility constraints in object tracks; and combining point track association and soft object track modeling to improve 3D localization accuracy.

2 . The method of claim 1 , comprising applying physical insights to model occlusion relationships.

3 . The method of claim 1 , comprising applying an occupancy model for traffic participants (TPs) that treats the TPs as translucent objects reflected by an occupancy function.

4 . The method of claim 1 , comprising applying a reflection model for handling visible points.

5 . The method of claim 1 , comprising applying a transmission model for representing occlusions from all other TPs in a scene.

6 . The method of claim 1 , comprising applying models to determine soft assignments of point tracks to TPs.

7 . The method of claim 1 , comprising applying models to account for occlusion relationships in object detection scores.

8 . The method of claim 1 , comprising applying 3D localization using the energies from point
tracks and detection models, along with energies that incorporate transition and collision constraints.

9 . The method of claim 1 , comprising treating objects as translucent entities.

10 . The method of claim 9 , comprising modelling objects as translucent 3D ellipsoids whose opacity is maximum at the center and falls off towards the edges, further comprising modeling occupancy at location x corresponding to a traffic participant-centered at p as:
ƒ occ ( x )= L ( x;P ,Σ)
where L(•) is a logistic function given by

L


(

x
;
p

,
Σ

)

=

1

1
+



-

k


(

1
-

d


(

x
,
p

)

)

,

with d(x,p)=(x−p) T Σ(x−p) being a Mahalanobis distance.

11 . The method of claim 1 , comprising defining an image formation model that accounts for reflection and transmission as a soft way to model occlusions.

12 . The method of claim 1 , comprising defining a reflection probability model that determines when a point is visible in the image.

13 . The method of claim 1 , comprising defining transmission probability model proposes a soft way to deal with occlusions due to intermediate objects in the path of a back-projected ray.

14 . The method of claim 1 , comprising defining a point tracks association energy to assign points in a soft fashion to various traffic participants.

15 . The method of claim 1 , comprising defining an object tracks energy to probabilistically adjust detection bounding boxes and scores to account for occlusions.

16 . The method of claim 1 , comprising defining energy by a dot product between car orientation and tangent to a lane at a point.

17 . The method of claim 16 , comprising determining

E
lane
it

=

∑

m
∈

M
close





(

1
-

ω
i



(
t
)

·

TAN


(

L
m



(
k
)

,

p
i



(
t
)

)

)



Σ

L
m



(

p
i



(
t
)

Assignees

• Nec Lab America Inc

Inventors

• Chandraker Manmohan
• Dhiman Vikas

Classifications

• H04N23/00

  Cameras or camera modules comprising electronic image sensors; Control thereof · CPC title

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• H04N7/183

  for receiving images from a single remote source · CPC title

• B60R2300/108

  using 'non-standard' camera systems, e.g. camera sensor used for additional purposes i.a. rain sensor, camera sensor split in multiple image areas · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title