Range detection using machine learning combined with camera focus

Patent metadata

Field	Value
Publication number	US-11935258-B2
Application number	US-202117195368-A
Country	US
Kind code	B2
Filing date	Mar 8, 2021
Priority date	Mar 8, 2021
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

A method for range detection is described. The method includes segmenting an image into one or more segmentation blobs captured by a monocular camera of an ego vehicle. The method includes focusing on pixels forming a selected segmentation blob of the one or more segment blobs. The method also includes determining a distance to the selected segmentation blob according to a focus function value of the monocular camera of the ego vehicle.

First claim

What is claimed is:

1. A method for range detection, comprising:
segmenting an image into one or more segmentation blobs captured by a monocular camera of an ego vehicle, in which each segmentation blob of the one or more segmentation blobs comprises a collection of pixels associated with one respective object of one or more objects in a scene surrounding the ego vehicle;
performing, by a monocular camera, an autofocus operation to focus the monocular camera on the collection of pixels forming a selected segmentation blob of the one or more segment blobs based on an autofocus function value used by the monocular camera during the autofocus operation;
determining, according to the autofocus function value received from the monocular camera, a distance to the selected segmentation blob;
transmitting the determined distance and subsequent determined distances to the selected segmentation blob to a downstream advanced driver assistance system (ADAS);
identifying, by the ADAS, an object represented by the collection of pixels forming the selected segmentation blob as a moving vehicle according to the determined distance to the selected segmentation blob received from the monocular camera;
tracking, by the ADAS, a trajectory of the moving vehicle according to the subsequent determined distances to the moving vehicle received from the monocular camera according to subsequent autofocus function values used by the monocular camera to capture the moving vehicle; and
planning and executing a change in a trajectory of the ego vehicle based on the tracking of the trajectory of the moving vehicle with the ADAS.

2. The method of claim 1 , further in which focusing comprises adjusting a focus on the pixels forming the selected segmentation blob until a contrast of the pixels is at a predetermined value.

3. The method of claim 1 , further in which focusing comprises adjusting a focus on the pixels forming the selected segmentation blob until a phase detection of the pixels is at a predetermined value.

4. The method of claim 3 , in which the phase detection comprises dividing an incoming light into pairs of images and comparing the pairs of images.

5. The method of claim 1 , in which segmenting comprises semantic segmentation by detecting each pixel belonging to a class of the selected segmentation blob.

6. The method of claim 1 , in which segmenting comprises instance segmentation by identifying each pixel belonging to an instance of the selected segmentation blob.

7. A non-transitory computer-readable medium having program code recorded thereon for range detection, the program code being executed by a processor and comprising:
program code to segment an image into one or more segmentation blobs captured by a monocular camera of an ego vehicle, in which each segmentation blob of the one or more segmentation blobs comprises a collection of pixels associated with one respective object of one or more objects in a scene surrounding the ego vehicle;
program code to perform, by a monocular camera, an autofocus operation to focus the monocular camera on pixels forming a selected segmentation blob of the one or more segment blobs based on an autofocus function value used by the monocular camera during the autofocus operation;
program code to determine, according to the focus function value received from the monocular camera, a distance to the selected segmentation blob;
program code to transmit the determined distance and subsequent determined distances to the selected segmentation blob to a downstream advanced driver assistance system (ADAS);
program code to identify, by the ADAS, an object represented by the collection of pixels forming the selected segmentation blob as a moving vehicle according to the determined distance to the selected segmentation blob received from the monocular camera;
program code to track, by the ADAS, a trajectory of the moving vehicle according to the subsequent determined distances to the moving vehicle received from the monocular camera according to subsequent autofocus function values used by the monocular camera to capture the moving vehicle; and
program code to plan and execute a change in a trajectory of the ego vehicle based on the tracking of the trajectory of the moving vehicle with the ADAS.

8. The non-transitory computer-readable medium of claim 7 , in which the program code to focus comprises program code to adjust the focus on the pixels forming the selected segmentation blob until a contrast of the pixels is at a predetermined value.

9. The non-transitory computer-readable medium of claim 7 , in which the program code to focus comprises program code to adjust the focus on the pixels forming the selected segmentation blob until a phase detection of the pixels is at a predetermined value.

10. The non-transitory computer-readable medium of claim 9 , in which the phase detection comprises program code to divide an incoming light into pairs of images and comparing the pairs of images.

11. The non-transitory computer-readable medium of claim 7 , in which the program code to segment comprises semantic segmentation by program code to detect each pixel belonging to a class of the selected segmentation blob.

12. The non-transitory computer-readable medium of claim 7 , in which the program code to segment comprises instance segmentation by program code to identify each pixel belonging to an instance of the selected segmentation blob.

13. A system on chip for range detection, the system on chip comprising:
a segmentation blob detection module to segment an image into one or more segmentation blobs captured by a monocular camera of an ego vehicle, in which each segmentation blob of the one or more segmentation blobs comprises a collection of pixels associated with one respective object of one or more objects in a scene surrounding the ego vehicle;
a segmentation blob focus module to perform, by a monocular camera, an autofocus operation to focus the monocular camera on the collection of pixels forming a selected segmentation blob of the one or more segment blobs based on an autofocus function value used by the monocular camera during the autofocus operation;
a range detection module to determine, according to the focus function value received from the monocular camera, a distance to the selected segmentation blob and to transmit the determined distance and subsequent determined distances to the selected segmentation blob to a downstream advanced driver assistance system (ADAS);
the ADAS including a vehicle perception module to identify an object represented by the collection of pixels forming the selected segmentation blob as a moving vehicle according to the determined distance to the selected segmentation blob received from the monocular camera and the ADAS to track a trajectory of the moving vehicle according to received the subsequent determined distances to the moving vehicle received from the monocular camera according to subsequent autofocus function values used by the monocular camera to capture the moving vehicle; and
a trajectory planning module to plan and execute a change in a trajectory of the ego vehicle based on the determined distance to the moving vehicle and the tracking of the trajectory of the moving vehicle with the ADAS.

14. The system on chip of claim 13 , in which the segmentation blob focus module is further configured to adjust the focus on the pixels forming the selected segmentation blob until a contrast of the pixels is at a predetermined value.

15. The system on chip of claim 13 , in which the segmentation blob focus module is further configured to adjust the focus on the p

Assignees

• Toyota Res Inst Inc

Inventors

• Green Alexander Russell

Classifications

• G06T7/571 Primary

  from focus · CPC title

• B60W60/001

  Planning or execution of driving tasks · CPC title

• G01C21/3407

  specially adapted for specific applications · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title