Apparatus for controlling vehicle including camera and method for the same

Patent metadata

Field	Value
Publication number	US-12561985-B2
Application number	US-202217986091-A
Country	US
Kind code	B2
Filing date	Nov 14, 2022
Priority date	Mar 30, 2022
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

An apparatus of controlling a vehicle and a method for controlling the same includes a camera, a processor operatively connected to the camera, and a storage operatively connected to the processor to store instructions executed by the processor. The processor obtains an image of a surrounding of a door of the vehicle, which is captured through the camera, obtains depth information from the obtained image, transforms the depth information into three dimensional (3D) point information, determines collision possibility of the door with an obstacle, based on distribution of the 3D point information to determine collision possibility, and warns the collision of the door when the collision possibility is present, when executing the instructions.

First claim

What is claimed is:

1 . An apparatus of controlling a vehicle, the apparatus comprising:
a camera; a processor operatively connected to the camera; and a storage operatively connected to the processor and configured to store instructions executed by the processor, wherein, when executing the instructions, the processor is configured to:
obtain an image of a surrounding of a door of the vehicle, which is captured through the camera;
obtain depth information from the obtained image;
transform the depth information into three dimensional (3D) point information;
determine collision possibility of the door by determining the collision possibility of the door with an obstacle, based on distribution of the 3D point information; and
warn collision of the door, when the collision possibility is determined as being present, and
wherein, when executing the instructions, the processor is configured to:
set a collision sensing zone, having a predetermined size, wherein an endpoint of the collision sensing zone is spaced from a 3D point corresponding to a surface of the door by an offset;
determine a number of 3D points corresponding to the obstacle entering the collision sensing zone;
determine a variation in the number of 3D points corresponding to the obstacle entering the collision sensing zone during a period of time when a location of the collision sensing zone changes due to opening of the door; and
warn collision of the door, when the variation is equal to or greater than a predetermined number.

2 . The apparatus of claim 1 , wherein, when executing the instructions, the processor is configured to:
obtain the 3D point information by use of the depth information and a parameter of the camera, and wherein the parameter includes a focal length and coordinates of a principal point.

3 . The apparatus of claim 2 , wherein, when executing the instructions, the processor is configured to:
obtain the 3D point information through following Equation 1, Equation 2, and Equation 3,
x =(( px−cx )/ fx ) pz,   Equation 1,

y =(( py−cy )/ fy ) pz , and  Equation 2,

z=pz , and  Equation 3,

wherein ‘x’, ‘y’, and ‘z’ represent coordinates of a 3D point, ‘px’ and ‘py’ represent coordinates of each pixel contained in the depth information, ‘cx’ and ‘cy’ represent coordinates of the principal point, ‘fx’ and ‘fy’ represent the focal length, and ‘pz’ represents a depth value predicted with respect to the pixel.

4 . The apparatus of claim 1 , wherein, when executing the instructions, the processor is configured to:
produce a lattice having the predetermined size, based on the 3D point corresponding to the surface of the door; and set, as the collision sensing zone, a space defined from a point, which is spaced from the 3D point corresponding to the surface of the door by the offset, to a point spaced from the 3D point corresponding to the surface of the door by a predetermined sensing range, in a space corresponding to the lattice.

5 . The apparatus of claim 4 , wherein, when executing the instructions, the processor is configured to:
produce a plurality of lattices having the predetermined size based on a plurality of 3D points corresponding to the surface of the door; and warn the collision of the door, when 3D points corresponding to the obstacle are present in at least the predetermined number in the collision sensing zone of at least one lattice of the plurality of lattices.

6 . The apparatus of claim 1 , wherein, when executing the instructions, the processor is configured to:
determine the collision possibility of the door as being present, when the variation is equal to or greater than a predetermined value.

7 . The apparatus of claim 1 ,
wherein the vehicle control apparatus further includes a speaker, and wherein, when executing the instructions, the processor is configured to: output a warning sound through the speaker, when determining the collision possibility as being present.

8 . The apparatus of claim 1 , wherein the camera includes:
at least one of a mono-camera or a stereo-camera.

9 . A method for controlling a vehicle, the method comprising:
obtaining, by a processor, an image of a surrounding of a door of the vehicle, which is captured through a camera; obtaining, by the processor, depth information from the obtained image; transforming, by the processor, the depth information into 3D point information; determining, by the processor, collision possibility of the door by determining the collision possibility of the door with an obstacle, based on distribution of the 3D point information; and warning, by the processor, collision of the door, when the collision possibility is determined as being present, wherein the determining of the collision possibility includes:
setting a collision sensing zone, having a predetermined size, wherein an endpoint of the collision sensing zone is spaced from a 3D point corresponding to a surface of the door by an offset;
determining a number of 3D points corresponding to the obstacle entering the collision sensing zone,
determine a variation in the number of 3D points corresponding to the obstacle entering the collision sensing zone during a period of time when a location of the collision sensing zone changes due to opening of the door, and
determining the collision possibility of the door, based on the variation.

10 . The method of claim 9 , wherein the transforming of the depth information into the 3D point information includes:
obtaining the 3D point information by use of the depth information and a parameter of the camera, and wherein the parameter includes a focal length and coordinates of a principal point.

11 . The method of claim 10 , wherein the obtaining of the 3D point information is performed through following Equation 1, Equation 2, and Equation 3,
x =(( px−cx )/ fx )· pz,   Equation 1,
y =(( py−cy )/ fy )· pz , and  Equation 2,
z=pz,   Equation 3,
wherein ‘x’, ‘y’, and ‘z’ represent coordinates of a 3D point, ‘px’ and ‘py’ represent coordinates of each pixel contained in the depth information, ‘cx’ and ‘cy’ represent coordinates of a principal point, ‘fx’ and ‘fy’ represent focal lengths, and ‘pz’ represents a depth value predicted with respect to the pixel.

12 . The method of claim 9 , wherein the setting of the collision sensing zone includes:
producing a lattice having the predetermined size based on the 3D point corresponding to the surface of the door; and setting, as the collision sensing zone, a space defined from a point, which is spaced from the 3D point corresponding to the surface of the door by the offset, to a point spaced from the 3D predetermined point corresponding to the surface of the door by a predetermined sensing range, in a space corresponding to the lattice.

13 . The method of claim 12 , wherein the producing of the lattice includes:
producing a plurality of lattices having the predetermined size based on a plurality of 3D points corresponding to the surface of the door, and wherein the warning of the collision of the door includes: determining whether 3D points corresponding to the obstacle are present in at least the predetermined number in the collision sensing zone of at least one lattice of the plurality of lattices.

14 . The method of claim 9 , wherein the determining of the collision possibility includes:
determining the collision possibility of the door as being present, when the variation is equal to or greater than a predetermined value.

15 . The method of claim 9 , wherein the warning of the collision o

Assignees

• Hyundai Motor Co Ltd
• Kia Corp
• Industry Univ Cooperation Foundation Hanyang Univ Iucf Hyu

Inventors

• Yim Chung Sik
• Kim Soo Bok
• Yun Hyung In
• Choi Jun Won
• Koh Jun Ho
• Lee Young Woo
• Yoon In Young

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30261

  Obstacle · CPC title

• B60Q9/008

  for anti-collision purposes · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/50

  Depth or shape recovery · CPC title