Method for detecting at least one obstacle in an automated and/or at least semi-autonomous driving system

Patent metadata

Field	Value
Publication number	US-2024393804-A1
Application number	US-202418662291-A
Country	US
Kind code	A1
Filing date	May 13, 2024
Priority date	May 26, 2023
Publication date	Nov 28, 2024
Grant date	—

Abstract

The invention relates to method ( 100 ) for detecting at least one obstacle in an automated and/or at least semi-autonomous driving system ( 60 ), said method comprising the following steps:
providing ( 101 ) image data, wherein the image data are specific to a recording of an environment of the driving system ( 60 ), performing ( 102 ) an evaluation of the image data provided, wherein the evaluation takes place based on an application of a machine learning model ( 50 ), by means of which an occlusion label is determined for at least one occlusion of the environment, performing ( 103 ) the detection of the at least one obstacle on the basis of the occlusion label determined.

First claim

1 . A method for detecting at least one obstacle in an automated and/or at least semi-autonomous driving system, said method comprising the following steps:
providing image data, wherein the image data are specific to a recording of an environment of the driving system, performing an evaluation of the image data provided, wherein the evaluation takes place based on an application of a machine learning model, by means of which an occlusion label is determined for at least one occlusion of the environment, performing the detection of the at least one obstacle on the basis of the occlusion label determined.

2 . The method according to claim 1 , characterized in that
training of the machine learning model is based on an occlusion area being determined on the basis of a movement in a camera recording, wherein an optical flow is preferably estimated for this purpose in a sequence of images resulting from the camera recording, and the machine learning model is trained in reference to the estimated optical flow in order to determine the occlusion label, wherein the training is preferably performed in the form of a self-supervised training process.

3 . The method according to claim 1 , characterized in that
the image data, in particular in inference mode, comprise at least one or exactly one individual image, which results from a recording by means of a monocular or stereo camera, wherein the image data used for the machine learning model as input for determining the occlusion label are preferably limited to the individual image.

4 . The method according to claim 1 , characterized in that
the occlusion label is specific to the at least one occlusion and is preferably designed as a occlusion map which identifies at least one or multiple areas in the image data that are occluded by at least one object in the environment.

5 . The method according to claim 1 ,
characterized in that the detection of the at least one obstacle comprises an evaluation of the occlusion label, preferably by means of a classifier, during which evaluation a classification of one of the objects, which is in the form of a hazardous object associated with the respective occlusion and detected in the image data, is performed in reference to the occlusion label, wherein the hazardous object in particular comprises cargo that has fallen from a truck.

6 . The method according to claim 1 , characterized in that, based on the evaluation and/or detection, at least partially autonomous control of an ego vehicle and/or a robot is performed by the driving system, preferably by a motion planning system.

7 . A training method for training a machine learning model, said method comprising:
providing training data, wherein the training data comprise at least one sequence of images representing an environment of a driving system during a trip, wherein the training data further comprise annotation data which indicate an occlusion label representing at least one occlusion of the environment during the trip, performing training of the machine learning model on the basis of the training data, during which training an optical flow in the sequence of images is taken into account in order to predict the occlusion label.

8 . A computer program comprising instructions which, when the computer program is executed by a computer, prompt the latter to:
provide image data, wherein the image data are specific to a recording of an environment of the driving system, perform an evaluation of the image data provided, wherein the evaluation takes place based on an application of a machine learning model, by means of which an occlusion label is determined for at least one occlusion of the environment, and perform the detection of the at least one obstacle on the basis of the occlusion label determined.

9 . A device for data processing, which is configured to:
provide image data, wherein the image data are specific to a recording of an environment of the driving system, perform an evaluation of the image data provided, wherein the evaluation takes place based on an application of a machine learning model, by means of which an occlusion label is determined for at least one occlusion of the environment, and perform the detection of the at least one obstacle on the basis of the occlusion label determined.

10 . A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, prompt the latter to perform the method according to
provide image data, wherein the image data are specific to a recording of an environment of the driving system, perform an evaluation of the image data provided, wherein the evaluation takes place based on an application of a machine learning model, by means of which an occlusion label is determined for at least one occlusion of the environment, and perform the detection of the at least one obstacle on the basis of the occlusion label determined.

11 . A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, prompt the latter to;
provide training data, wherein the training data comprise at least one sequence of images representing an environment of a driving system during a trip, wherein the training data further comprise annotation data which indicate an occlusion label representing at least one occlusion of the environment during the trip, performing training of a machine learning model on the basis of the training data, during which training an optical flow in the sequence of images is taken into account in order to predict the occlusion label.

Assignees

• Bosch Gmbh Robert
• Cariad Se

Inventors

• Tananaev Denis
• Backfish Niels

Classifications

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title