Apparatus for autonomous driving and method and system for calibrating sensor thereof

Patent metadata

Field	Value
Publication number	US-11587256-B2
Application number	US-202017110853-A
Country	US
Kind code	B2
Filing date	Dec 3, 2020
Priority date	Dec 5, 2019
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

The autonomous driving device including a communication circuit configured to communicate with an unmanned aerial vehicle, a plurality of sensors disposed in the autonomous vehicle to monitor all directions of the autonomous vehicle, and a processor, wherein the processor is configured to: control the unmanned aerial vehicle to hover at each of a plurality of waypoints of a designated flight path by controlling a relative position of the unmanned aerial vehicle through the communication circuit, change a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path, generate a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through the plurality of sensors, and calibrate the plurality of sensors on the basis of a relationship between matching points of the plurality of images.

First claim

What is claimed is:

1. An autonomous driving device included in an autonomous vehicle, the autonomous driving device comprising:
a communication circuit configured to communicate with an unmanned aerial vehicle to which a checkerboard is fixed;
a plurality of sensors disposed in the autonomous vehicle to monitor all directions of the autonomous vehicle; and
a processor,
wherein the processor is configured to:
control the unmanned aerial vehicle to hover at each of a plurality of waypoints of a designated flight path by controlling a relative position of the unmanned aerial vehicle through the communication circuit;
change a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path;
generate a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through the plurality of sensors; and
calibrate the plurality of sensors on the basis of a relationship between matching points of the plurality of images.

2. The autonomous driving device of claim 1 , wherein the processor designates the flight path including the plurality of waypoints so that the unmanned aerial vehicle is positioned in a visible range of at least one of the plurality of sensors.

3. The autonomous driving device of claim 2 , wherein the processor designates the plurality of waypoints to include all regions in which visible ranges of two of the plurality of sensors overlap each other.

4. The autonomous driving device of claim 2 , wherein the processor designates, as the plurality of waypoints, intersections between a line spaced a specific distance from the autonomous vehicle and points obtained by dividing a circumference of the autonomous vehicle by first unit angles.

5. The autonomous driving device of claim 2 , wherein the processor is configured to:
designate the plurality of posture angles related to the waypoints so that the checkerboard and the autonomous vehicle form a plurality of angles ranging from a first critical angle to a second critical angle at the waypoints; and
associate the designated posture angles with the waypoints of the flight path.

6. The autonomous driving device of claim 2 , wherein
the processor stores the designated flight path in a memory when the flight path is designated, and
the processor uses the designated flight path during subsequent calibration of the plurality of sensors.

7. The autonomous driving device of claim 1 , wherein the processor updates the flight path on the basis of a result of the calibration of the plurality of sensors.

8. The autonomous driving device of claim 1 , wherein the processor is configured to:
determine an intrinsic parameter and an extrinsic parameter during the calibration of the plurality of sensors; and
determine position coordinates of a nearby object on the basis of the determined intrinsic parameter and the extrinsic parameter when the nearby object is detected through the plurality of sensors.

9. A sensor calibration system comprising:
an unmanned aerial vehicle to which a checkerboard is fixed; and
an autonomous driving device configured to control the unmanned aerial vehicle and included in an autonomous vehicle,
wherein the autonomous driving device is configured to:
control the unmanned aerial vehicle to hover at each of a plurality of waypoints of a designated flight path by controlling a relative position of the unmanned aerial vehicle;
change a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path;
generate a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through a plurality of sensors; and
calibrate the plurality of sensors on the basis of a relationship between matching points of the plurality of images.

10. The sensor calibration system of claim 9 , wherein
the unmanned aerial vehicle is fixed or housed in a region of the autonomous vehicle, and
the unmanned aerial vehicle is configured to start flight corresponding to the flight path according to a command of the autonomous driving device.

11. The sensor calibration system of claim 9 , wherein
the unmanned aerial vehicle changes a flight state, which is at least one of a flight position or a posture angle, according to a command received from the autonomous driving device, and
the unmanned aerial vehicle transmits flight state reporting information corresponding to a change of the at least one flight state to the autonomous driving device.

12. The sensor calibration system of claim 9 , wherein
the unmanned aerial vehicle comprises a Global Positioning System (GPS) module configured to detect current position coordinates of the unmanned aerial vehicle, and
the unmanned aerial vehicle controls a flight position according to a command received from the autonomous vehicle on the basis of the detected current position coordinates.

13. The sensor calibration system of claim 9 , wherein the autonomous driving device is configured to:
designate the flight path including the plurality of waypoints so that the unmanned aerial vehicle is positioned in a visible range of at least one of the plurality of sensors; and
control a flight position of the unmanned aerial vehicle according to the designated flight path.

14. The sensor calibration system of claim 9 , wherein the autonomous driving vehicle is configured to:
designate the plurality of posture angles related to the waypoints so that the checkerboard and the autonomous vehicle form a plurality of angles ranging from a first critical angle to a second critical angle at the waypoints, associate the designated posture angles with the waypoints of the flight path; and
control a posture angle of the unmanned aerial vehicle to be the plurality of posture angles at the waypoints.

15. A sensor calibration method performed by an autonomous driving device, the sensor calibration method comprising:
controlling an unmanned aerial vehicle, to which a checkerboard is fixed, to hover at each of a plurality of waypoints of a designated flight path;
changing a posture angle of the unmanned aerial vehicle to a plurality of posture angles corresponding to the waypoints of the flight path;
generating a plurality of images including the checkerboard and corresponding to the plurality of waypoints and the plurality of posture angles through a plurality of sensors included in the autonomous driving device; and
calibrating the plurality of sensors on the basis of a relationship between matching points of the plurality of images.

16. The sensor calibration method of claim 15 , further comprising designating the flight path including the plurality of waypoints so that the unmanned aerial vehicle is positioned in a visible range of at least one of the plurality of sensors.

17. The sensor calibration method of claim 16 , where the designating of the flight path comprises designating the plurality of waypoints to include all regions in which visible ranges of two of the plurality of sensors overlap each other.

18. The sensor calibration method of claim 16 , wherein the designating of the flight path comprises designating, as the plurality of waypoints, intersections between a curve spaced a specific distance from an autonomous vehicle and points obtained by dividing a circumference of the autonomous vehicle by first unit angles.

19. The sensor calibration method of claim 16 , whe

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Park Jae Hyuck
• Jo Yong Woo
• Choi Doo Seop
• Min Kyoung Wook
• Choi Jeong Dan

Classifications

• B64U2201/20

  Remote controls · CPC title

• B64U2201/104

  using satellite radio beacon positioning systems, e.g. GPS · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• B64U80/60

  by wearable objects, e.g. garments or helmets · CPC title

• B64U10/13

  Flying platforms · CPC title