Monocular camera system performing depth estimation of objects surrounding a vehicle

Patent metadata

Field	Value
Publication number	US-12125225-B1
Application number	US-202318295427-A
Country	US
Kind code	B1
Filing date	Apr 4, 2023
Priority date	Apr 4, 2023
Publication date	Oct 22, 2024
Grant date	Oct 22, 2024

Abstract

A monocular camera system for a vehicle includes a front mono camera, a side mono camera, and one or more controllers in electronic communication with the front mono camera and the side mono camera. The one or more controllers execute instructions to determine an ideal lagged distance the vehicle travels between a current time step and a previous time step as two asynchronous camera frames are captured by the front mono camera and the side mono camera, determine a number of delayed frames captured by either the front mono camera or the side mono camera between the current time step and the previous time step based on the ideal lagged distance. The controllers determine a direction of travel of the vehicle that indicates which mono camera is selected to provide a previous camera frame captured at the previous time step.

First claim

What is claimed is:

1. A monocular camera system for a vehicle, the monocular camera system comprising:
a front mono camera positioned at a front portion of the vehicle;
a side mono camera positioned along a side of the vehicle; and
one or more controllers in electronic communication with the front mono camera and the side mono camera, the one or more controllers executing instructions to:
collect image data captured by the front mono camera and the side mono camera, wherein the image data includes an object being evaluated;
determine an ideal lagged distance the vehicle travels between a current time step and a previous time step as two asynchronous camera frames are captured by the front mono camera and the side mono camera;
determine a number of delayed frames captured by either the front mono camera or the side mono camera between the current time step and the previous time step based on the ideal lagged distance;
determine a direction of travel of the vehicle, wherein the direction of travel of the vehicle indicates which mono camera is selected to provide a previous camera frame captured at the previous time step;
perform feature matching to identify pixel positions in a camera frame captured by the front mono camera and a camera frame captured by the side mono camera that correspond to the same three-dimensional coordinates of the object being evaluated; and
perform triangulation of the pixel positions captured by the front mono camera and the side mono camera to determine the three-dimensional coordinates of the object being evaluated.

2. The monocular camera system of claim 1 , wherein the one or more controllers execute instructions to:
in response to determining the vehicle is traveling in a forward direction, select the previous camera frame in the image data captured by the front mono camera at the previous time step and the camera frame captured by the side mono camera at the current time step.

3. The monocular camera system of claim 1 , wherein the one or more controllers execute instructions to:
in response to determining the vehicle is traveling in a backwards direction, select the previous camera frame in the image data captured by the side mono camera at the previous time step and the camera frame captured by the front mono camera at the current time step.

4. The monocular camera system of claim 1 , wherein the number of delayed frames indicates how many frames are counted back in the image data captured by either the front mono camera or the side mono camera between the current time step and the previous time step.

5. The monocular camera system of claim 1 , wherein the number of delayed frames is determined based on:

camera
⁢

frames

≅

floor
(

❘
"\[LeftBracketingBar]"

s

v
l

❘
"\[RightBracketingBar]"

×
FPS

)

wherein v l is a longitudinal velocity of the vehicle, s represents the ideal lagged distance, FPS represents frames per second of a subject camera, and floor( ) represents an operator that returns the integer part of a floating point number.

6. The monocular camera system of claim 1 , wherein determining the ideal lagged distance comprises:
solving for an area of overlap between a front field-of-view captured by the front mono camera and a side field-of-view captured by the side mono camera.

7. The monocular camera system of claim 6 , wherein a steering angle of the vehicle is zero, and the area of overlap is determined by:

A
∼

∫

x
r

l
e

(

L
1

(

x
r

,

y
r

)

-

L
2

)

⁢
dx

⇒

xr
~
cte

A
∼

∫

x
r

l
e

(

L
1

(

y
r

)

-

L
2

)

⁢
dx

wherein A represents the area of overlap, L 1 represents a line equation defining boundaries of a side field-of-view of the side mono camera, L 2 represents a line equation defining boundaries of a front field-of-view of the front mono camera, x r represents an x-position of the side mono camera, y r represents a y-position of the side mono camera, l e is an upper limit of the integration, x represent a lateral direction, and y represents the longitudinal direction.

8. The monocular camera system of claim 7 , wherein the line equation defining boundaries of the side field-of-view of the side mono camera is solved as:

L
1

=

{

Assignees

• Gm Global Tech Operations Llc
• Univ Waterloo

Inventors

• Alghooneh Ahmad R
• Khajepour Amir
• Alghooneh Mansoor
• Bacchus Brent Navin Roger
• Bhatt Neel Pratik
• Zhang Ruihe

Classifications

• G06T2207/30248

  Vehicle exterior or interior · CPC title

• B60R2011/004

  outside the vehicle · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06V10/74

  Image or video pattern matching; Proximity measures in feature spaces · CPC title