Control device and control method for mobile object, storage medium, and vehicle

Patent metadata

Field	Value
Publication number	US-11893715-B2
Application number	US-202217688084-A
Country	US
Kind code	B2
Filing date	Mar 7, 2022
Priority date	Mar 23, 2021
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

A control device for a mobile object including one or more imaging devices, the control device includes an image acquisition unit configured to acquire an image of an external environment of the mobile object from the one or more imaging devices, a correction unit configured to perform distortion reduction processing for reducing distortion of an image for each of one or more regions included in an image acquired from the one or more imaging devices, and a recognition unit configured to recognize the external environment of the mobile object based on an image on which the distortion reduction processing has been performed. The correction unit is configured to determine the one or more regions to be a target of the distortion reduction processing in accordance with a predetermined rule according to a moving scene of the mobile object.

First claim

What is claimed is:

1. A control device for a mobile object including one or more imaging devices, the control device comprising:
at least one processor circuit with a memory comprising instructions, that when executed by the processor circuit, cause the processor circuit to at least:
acquire an image of an external environment of the mobile object from the one or more imaging devices;
perform distortion reduction processing for reducing distortion of an image for each of one or more regions included in an image acquired from the one or more imaging devices, the distortion reduction processing including coordinate transformation of a target region; and
recognize the external environment of the mobile object based on an image on which the distortion reduction processing has been performed,
wherein the one or more regions is determined to be a target of the distortion reduction processing in accordance with a predetermined rule according to a moving scene of the mobile object,
the predetermined rule defines that for the moving scene:
images from the one or more imaging devices are subjected to the distortion reduction processing at a first frequency for a first region; and
images from the one or more imaging devices are subjected to the distortion reduction processing at a second frequency for a second region, the second frequency being higher than the first frequency, and

the instructions, that when executed by the processor circuit, cause the processor circuit to perform the distortion reduction processing to the images in the first frequency for the first region and in the second frequency for the second region when the mobile object is in the first moving scene.

2. The control device according to claim 1 , wherein
the instructions cause the processor circuit to perform the distortion reduction processing every predetermined cycle, and
the predetermined rule is a rule that defines a region to be the target of the distortion reduction processing every predetermined cycle.

3. The control device according to claim 1 , wherein
the predetermined rule defines
a position in a horizontal direction of a region to be the target of the distortion reduction processing, and
a timing at which a region at the position is set as the target of the distortion reduction processing.

4. The control device according to claim 1 , wherein the predetermined rule defines a position in a vertical direction of a region to be the target of the distortion reduction processing.

5. The control device according to claim 1 , wherein
the mobile object includes a plurality of imaging devices,
the instructions cause the processor circuit to acquire an image of the external environment of the mobile object from each of the plurality of imaging devices, and
the predetermined rule defines an individual rule for each of the plurality of imaging devices.

6. The control device according to claim 5 , wherein
the plurality of imaging devices includes
a first imaging device that captures in a direction directly ahead of the mobile object, in a direction right-diagonally ahead of the mobile object, and in a direction left-diagonally ahead of the mobile object,
a second imaging device that captures in a direction directly behind the mobile object, in a direction right-diagonally behind the mobile object, and in a direction left-diagonally behind the mobile object,
a third imaging device that captures in a direction directly right of the mobile object, in the direction right-diagonally ahead of the mobile object, and in the direction right-diagonally behind the mobile object, and
a fourth imaging device that captures in a direction directly left of the mobile object, in the direction left-diagonally ahead of the mobile object, and in the direction left-diagonally behind the mobile object.

7. The control device according to claim 1 , wherein the mobile object is a vehicle, and the moving scene is a traveling scene of the vehicle.

8. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle enters a T-junction or when the vehicle restarts after a temporary stop,
the direction left-diagonally ahead of the vehicle and the direction right-diagonally ahead of the vehicle are set as targets of the distortion reduction processing more frequently than the direction directly right of the vehicle, the direction directly left of the vehicle, the direction right-diagonally behind the vehicle, the direction left-diagonally behind the vehicle, and the direction directly behind the vehicle.

9. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle enters the T-junction or when the vehicle restarts after the temporary stop,
two regions in a direction diagonally ahead of the vehicle out of two images acquired from two imaging devices are set as targets of the distortion reduction processing at a same timing.

10. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle travels on a narrow road,
the direction left-diagonally ahead of the vehicle and the direction right-diagonally ahead of the vehicle are set as targets of the distortion reduction processing more frequently than the direction directly right of the vehicle, the direction directly left of the vehicle, the direction right-diagonally behind the vehicle, the direction left-diagonally behind the vehicle, and the direction directly behind the vehicle.

11. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle travels on a narrow road,
the direction right-diagonally ahead of the vehicle and the direction left-diagonally ahead of the vehicle are set as targets of the distortion reduction processing more frequently than the direction directly right of the vehicle, the direction directly left of the vehicle, the direction right-diagonally behind of vehicle, the direction left-diagonally behind the vehicle, and the direction directly behind the vehicle.

12. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle travels on a narrow road, as compared with a case where the vehicle travels on a path other than the narrow road,
positions in a vertical direction of regions in the direction directly right of the vehicle, in the direction directly left of the vehicle, in the direction right-diagonally behind the vehicle, in the direction left-diagonally behind the vehicle, and in the direction directly behind the vehicle are on a lower side.

13. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle travels on a narrow road, as compared with a case where the vehicle travels on a path other than the narrow road,
positions in a vertical direction of regions in the direction directly ahead of the vehicle, in the direction right-diagonally ahead of the vehicle, and in the direction left-diagonally ahead of the vehicle are on a lower side.

14. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle travels on a narrow road,
positions in a vertical direction of regions in the direction right-diagonally ahead of the vehicle and in the direction left-diagonally ahead of the vehicle are on a higher side than positions in the vertical direction of regions in the direction directly right of the vehicle and in the direction directly left of the vehicle.

15. The control device according to claim 7 , wherein
the predetermined rule defines that, when the vehicle tu

Assignees

• Honda Motor Co Ltd

Inventors

• Yasui Yuji
• Kanehara Akira
• Goto Kenji
• Miura Aya
• Baba Ichiro

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T5/006 Primary

  Physics · mapped topic

• B60W40/06

  Road conditions · CPC title

• B60W50/06

  Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot · CPC title

• G06V10/22

  by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition · CPC title