System and method for camera-based detection of object heights proximate to a vehicle

Patent metadata

Field	Value
Publication number	US-10699136-B2
Application number	US-201716073760-A
Country	US
Kind code	B2
Filing date	Jan 25, 2017
Priority date	Jan 29, 2016
Publication date	Jun 30, 2020
Grant date	Jun 30, 2020

Abstract

The present invention is generally directed to optical data processing or objects in proximity to an optical sensor or camera, such as a camera mounted on a vehicles. Image data is received from a camera mounted on a vehicle, the image data indicative of a motion of the camera with respect to a detected object within a proximity of the camera or the vehicle on which the camera may be mounted. An optical flow of the object in the image data is determined, and a height of the object is determined based on the determined optical flow.

First claim

The invention claimed is:

1. An optical processing system of a vehicle comprising:
one or more processors; and
a memory including instructions, which when executed by the one or more processors, cause the one or more processors to perform a method comprising the steps of:
receiving image data from a first camera of the vehicle, the image data indicative of a motion of the first camera with respect to a first object in proximity to the vehicle and indicative of a motion of the first camera with respect to a second object in proximity to the vehicle;
determining a first optical flow of the first object in the image data;
determining a first height of the first object based on the first optical flow;
determining a second optical flow, different from the first optical flow, of the second object in the image data;
determining a second height of the second object, different from the first height, based on the first optical flow;
identifying the first object in the image data as a curb based on the first optical flow; and
identifying the second object in the image data as an object on the curb based on the second optical flow.

2. The system of claim 1 , wherein said step of determining the first height of the first object based on the first optical flow further comprises the steps of:
comparing the first optical flow to the second optical flow of the second object in the image data;
in accordance with a determination that the first optical flow is less than the second optical flow, determining that the first height of the first object is less than the second height of the second object; and
in accordance with a determination that the first optical flow is greater than the second optical flow, determining that the first height of the first object is greater than the second height of the second object.

3. The system of claim 2 , wherein said step of determining the first height of the first object based on the first optical flow includes
determining the first height of the first object based on a difference between the first optical flow and the second optical flow.

4. The system of claim 1 , wherein said step of determining the first height of the first object based on the first optical flow comprises the steps of:
comparing the first optical flow to a predetermined optical flow, the predetermined optical flow being independent of the image data and corresponding to a predetermined height;
in accordance with a determination that the first optical flow is less than the predetermined optical flow, determining that the first height of the first object is less than the predetermined height; and
in accordance with a determination that the first optical flow is greater than the predetermined optical flow, determining that the first height of the first object is greater than the predetermined height.

5. The system of claim 4 , wherein said step of determining the first height of the first object based on the first optical flow includes
determining the first height of the first object based on a difference between the first optical flow and the predetermined optical flow.

6. The system of claim 1 , wherein the step of identifying the first object comprises the steps of:
comparing the first height of the first object to a predetermined height associated with a particular object, the predetermined height being independent of the image data; and
in accordance with a determination that the first height is within a predetermined height threshold of the predetermined height, identifying the first object as the particular object.

7. The system of claim 1 , wherein the step of identifying the first object comprises the steps of:
comparing the first optical flow of the first object to a predetermined optical flow associated with a particular object, the predetermined optical flow being independent of the image data; and
in accordance with a determination that the first optical flow is within a predetermined optical flow threshold of the predetermined optical flow, identifying the first object as the particular object.

8. The system of claim 1 , said method further comprising the step of determining a first location of the first object relative to the vehicle based on the image data.

9. The system of claim 8 , wherein said step of determining the first location of the first object is based on the image data and data from a second sensor, different from the first camera.

10. The system of claim 9 , wherein the second sensor comprises one of an ultrasonic sensor, a radar sensor, a laser sensor and a LIDAR sensor.

11. The system of claim 1 , said method further comprising the step of performing autonomous driving or parking operations with the vehicle based on the determined first height.

12. A non-transitory computer-readable medium including instructions, which when executed by one or more processors, cause the one or more processors to perform a method comprising:
receiving image data from a first camera, the image data indicative of a motion of the first camera with respect to a first object in proximity to the first camera and indicative of a motion of the first camera with respect to a second object in proximity to the first camera;
determining a first optical flow of the first object in the image data; and
determining a first height of the first object relative to the first camera based on the first optical flow
determining a second optical flow, different from the first optical flow, of the second object in the image data;
determining a second height of the second object, different from the first height, based on the first optical flow;
identifying the first object in the image data as a curb based on the first optical flow; and
identifying the second object in the image data as an object on the curb based on the second optical flow.

13. A vehicle comprising:
a first camera;
one or more processors coupled to the first camera; and
a memory including instructions, which when executed by the one or more processors, cause the one or more processors to perform a method comprising:
receiving image data from the first camera, the image data indicative of a motion of the first camera with respect to a first object in an environment of the vehicle and indicative of a motion of the first camera with respect to a second object in the environment of the vehicle;
determining a first optical flow of the first object in the image data; and
determining a first height of the first object based on the first optical flow;
determining a second optical flow, different from the first optical flow, of the second object in the image data;
determining a second height of the second object, different from the first height, based on the first optical flow;
identifying the first object in the image data as a curb based on the first optical flow; and
identifying the second object in the image data as an object on the curb based on the second optical flow.

Assignees

• Faraday&Future Inc
• Faraday & Future Inc

Inventors

• Theodosis Paul Alan
• Jeromin Oliver Max

Classifications

• G06V20/586 Primary

  of parking space · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06K9/00805 Primary

  Physics · mapped topic

• G06K9/00812

  Physics · mapped topic