Method and apparatus for image processing for vehicle

Patent metadata

Field	Value
Publication number	US-11017270-B2
Application number	US-201916566824-A
Country	US
Kind code	B2
Filing date	Sep 10, 2019
Priority date	Sep 5, 2017
Publication date	May 25, 2021
Grant date	May 25, 2021

Abstract

A method and an apparatus for image processing for a vehicle are provided. The method includes: acquiring an image presenting a target object; occluding at least one area of the image to obtain at least one occluded image; inputting the at least one occluded image into a pre-trained image processing model to obtain a driving parameter corresponding to each of the at least one occluded image; determining, for each driving parameter, a difference between the driving parameter and a real driving parameter corresponding to the acquired image; and determining whether the difference between the driving parameter of the occluded image occluding at least one area where the target object is located and the real driving parameter is greater than or equal to a difference threshold, to determine a degree of association between the state of the target object and the driving parameter.

First claim

What is claimed is:

1. A method for image processing for a vehicle, the method comprising:
acquiring an image presenting a target object;
occluding at least one area of the acquired image to obtain at least one occluded image;
inputting the at least one occluded image into a pre-trained image processing model to obtain a driving parameter corresponding to each of the at least one occluded image, the pre-trained image processing model being configured to characterize a corresponding relationship between a state of the target object presented in the image and the obtained driving parameter of the vehicle;
determining, for each obtained driving parameter, a difference between the obtained driving parameter and a real driving parameter corresponding to the acquired image, wherein the real driving parameter is a driving parameter that is used by a driver when encountering the target object of the acquired image when driving the vehicle in a real scenario that the driver is driving; and
determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to a difference threshold, to determine a degree of association between the state of the target object and the driving parameter, and to determine a learning status of the image processing model on a feature of the state of the target object.

2. The method for image processing for the vehicle according to claim 1 , wherein the driving parameter comprises a steering wheel angle and/or a driving speed.

3. The method for image processing for the vehicle according to claim 1 , wherein the method further comprises:
generating a thermodynamic diagram denoting the difference based on the difference between the driving parameter corresponding to the occluded image for each of the at least one occluded area of the acquired image and the real driving parameter.

4. The method for image processing for the vehicle according to claim 1 , wherein determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to the difference threshold, to determine the degree of association between the state of the target object and the driving parameter comprises:
determining the degree of association between the state of the target object and the driving parameter being strong, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being greater than or equal to the difference threshold; and
determining the degree of association between the state of the target object and the driving parameter being weak, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being smaller than the difference threshold.

5. The method for image processing for the vehicle according to claim 1 , wherein before inputting the at least one occluded image into the pre-trained image processing model to obtain the driving parameter corresponding to each of the at least one occluded image, the method further comprises:
training a pre-established image processing model based on an end to end approach to obtain the pre-trained image processing model;
extracting a feature of the target object from at least one image presenting the target object; and
adding the extracted feature of the target object to a feature database of the pre-trained image processing model.

6. An image processing apparatus applied to a vehicle, the apparatus comprising:
at least one processor; and
a memory storing instructions, wherein the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
acquiring an image presenting a target object;
occluding at least one area of the acquired image to obtain at least one occluded image;
inputting the at least one occluded image into a pre-trained image processing model to obtain a driving parameter corresponding to each of the at least one occluded image, the pre-trained image processing model being configured to characterize a corresponding relationship between a state of the target object presented in the image and the obtained driving parameter of the vehicle;
determining, for each obtained driving parameter, a difference between the obtained driving parameter and a real driving parameter corresponding to the acquired image, wherein the real driving parameter is a driving parameter that is used by a driver when encountering the target object of the acquired image when driving the vehicle in a real scenario that the driver is driving; and
determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to a difference threshold, to determine a degree of association between the state of the target object and the driving parameter and to determine a learning status of the image processing model on a feature of the state of the target object.

7. The image processing apparatus applied to the vehicle according to claim 6 , wherein the driving parameter comprises a steering wheel angle and/or a driving speed.

8. The image processing apparatus applied to a vehicle according to claim 6 , wherein the operations further comprise:
generating a thermodynamic diagram denoting the difference based on the difference between the driving parameter corresponding to the occluded image for each of the at least one occluded area of the acquired image and the real driving parameter.

9. The image processing apparatus applied to a vehicle according to claim 6 , wherein determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to the difference threshold, to determine the degree of association between the state of the target object and the driving parameter comprises:
determining the degree of association between the state of the target object and the driving parameter being strong, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being greater than or equal to the difference threshold; and
determining the degree of association between the state of the target object and the driving parameter being weak, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being smaller than the difference threshold.

10. The image processing apparatus applied to a vehicle according to claim 6 , wherein the operations further comprise:
training a pre-established image processing model based on an end to end approach to obtain the pre-trained image processing model;
extracting a feature of the target object from at least one image presenting the target object; and
adding the extracted feature of the target object to a feature database of the pre-trained image processing model.

11. A non-transitory computer readable storage medium, storing a computer program thereon, wherein the computer program, when executed by a pr

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Tang Kun
• Yu Hao
• Yan Yongshan
• Zheng Chao
• Zhang Yunfei
• Jiang Yu

Classifications

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06F18/214 Primary

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title