Aerial vehicle imaging and targeting system

Patent metadata

Field	Value
Publication number	US-11064184-B2
Application number	US-201816027683-A
Country	US
Kind code	B2
Filing date	Jul 5, 2018
Priority date	Aug 25, 2017
Publication date	Jul 13, 2021
Grant date	Jul 13, 2021

Abstract

The subject disclosure relates to a tracking system to mount to an aircraft and to image and track a target aircraft. The tracking system may include a structured light source operatively coupled to a processor, an inertial measurement unit (IMU) operatively coupled with the processor, a mirror to steer light from the light source toward the target aircraft, and a stereo-vision system having a first camera and a second camera. The IMU may be configured to generate position data representing a position of the aircraft. The stereo-vision system may be operatively coupled to the processor and configured to determine a 3D position of the target aircraft as a function of the position data. The processor may be configured to adjust the mirror position as a function of a mirror position.

First claim

What is claimed is:

1. A tracking system to mount to a vehicle and to image and track a moving object in an area, the tracking system comprising:
a structured light source operatively coupled to a processor;
an inertial measurement unit (IMU) operatively coupled with the processor, wherein the IMU is configured to generate position data representing a position of the vehicle;
a mirror to steer light from the light source toward the moving object; and
a stereo-vision system having a first camera and a second camera, the stereo-vision system being operatively coupled to the processor and configured to generate image data of the area at a refresh rate that is at least 800 HZ,
wherein the processor is configured to modulate the structured light source with a signal synchronized to the stereo-vision system to capture of the moving object with the light and without the light,
wherein the processor is configured to identify the moving object by comparing the images of the moving object and removing a background portion of the images,
wherein the stereo-vision system is configured to determine a three-dimensional position of the moving object as a function of both the position data of the vehicle and the image data, and
wherein the processer is configured to determine a predicted three-dimensional position for the moving object as a function of both the position data of the vehicle and the three-dimensional position of the moving object.

2. The tracking system of claim 1 , wherein the mirror is configured to direct the light from the light source toward the moving object as a function of a mirror position, wherein the processor is configured to adjust the mirror position.

3. The tracking system of claim 1 , wherein the structured light source is a laser light source configured to generate light with a wavelength in the 450-495 nm light spectrum.

4. The tracking system of claim 1 , wherein the structured light source is a laser light source configured to generate light with a wavelength in the blue light spectrum.

5. The tracking system of claim 4 , wherein the stereo-vision system includes a bandpass filter.

6. The tracking system of claim 1 , wherein the mirror is an electro-mechanical systems (MEMS) mirror.

7. The tracking system of claim 1 , wherein the first camera provides a first field of view (FOV) and the second camera provides a second FOV.

8. The tracking system of claim 7 , wherein the structured light source is configured to generate light that overlaps with at least a portion of each of the first FOV and the second FOV.

9. The tracking system of claim 1 , wherein the processor is configured to control positions of each of the mirror, the first camera, and the second camera based at least in part on the position data from the IMU.

10. The tracking system of claim 1 , wherein the vehicle is an aircraft and the moving object is an unmanned aerial vehicle.

11. A method of imaging and tracking a moving object using a sensor payload mounted to a vehicle, the method comprising the steps of:
generating, via an inertial measurement unit (IMU) operatively coupled with a processor, position data reflecting a position of the vehicle;
generating light via a structured light source operatively coupled to the processor;
steering the light from the structured light source toward the moving object via a mirror as a function of a mirror position, wherein the processor is configured to adjust the mirror position;
modulating the structured light source, via the processor, with a signal synchronized to a stereo-vision system;
capturing images of the moving object with the light and without the light at a refresh rate that is at least 800 Hz;
comparing the images of the moving object to generate image data, wherein a background portion in the images is removed to identify the moving object;
determining, via the stereo-vision system coupled to the processor, a three-dimensional position of the moving object as a function of the position data of the vehicle and the image data, wherein the stereo-vision system comprises a first camera and a second camera; and
determining, via the processor, a predicted three-dimensional position for the moving object as a function of the position data and the three-dimensional position.

12. The method of claim 11 , wherein the structured light source is a laser light source configured to generate light with a wavelength in the 450-495 nm light spectrum.

13. The method of claim 11 , wherein the structured light source is a laser light source configured to generate light with a wavelength in the blue light spectrum.

14. The method of claim 11 , wherein the mirror is an electro-mechanical systems (MEMS) mirror.

15. The method of claim 11 , wherein the first camera provides a first field of view (FOV) and the second camera provides a second FOV, and the light illuminates an area that overlaps with at least a portion of each of the first FOV and the second FOV.

16. The method of claim 15 , wherein at least a portion of the first FOV overlaps with a portion of the second FOV.

17. The method of claim 11 , wherein the processor is configured to control a position of each of the mirror, the first camera, and the second camera based at least in part on the position data from the IMU.

18. An aircraft to image and track a moving object, the aircraft comprising:
an airframe;
a structured light source mounted to the airframe and operatively coupled to a processor;
an inertial measurement unit (IMU) operatively coupled with the processor, wherein the IMU is configured to generate position data representing a position of the aircraft;
a mirror to steer light from the light source as a function of a mirror position, wherein the processor is configured to adjust the mirror position; and
a stereo-vision system having a first camera and a second camera, the stereo-vision system being mounted to the airframe and operatively coupled to the processor,
wherein stereo-vision system is configured to operate with a refresh rate that is at least 800 Hz,
wherein stereo-vision system is configured to modulate, via the processor, the light source with a signal synchronized to the stereo-vision system to capture images of the moving object with the light and without light,
wherein the stereo-vision system is configured to identify, via the processor, the moving object by comparing the images of the moving object and removing a background portion of the images,
wherein the stereo-vision system is configured to determine a three-dimensional position of the moving object relative to the aircraft,
wherein the processer is configured to determine a predicted three-dimensional position for the moving object as a function of the position data and the three-dimensional position.

19. The aircraft of claim 18 , wherein the mirror is configured to direct the light from the light source toward the moving object based at least in part on the position data from the IMU.

20. The aircraft of claim 18 , wherein the processor is configured to control a position of the mirror based on input from an acoustic sensor equipped to the aircraft.

Assignees

• Aurora Flight Sciences Corp

Inventors

• Choi Jae-Woo
• Paduano James D

Classifications

• B64U2101/30

  for imaging, photography or videography · CPC title

• H04N13/239 Primary

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• G03B15/006

  Apparatus mounted on flying objects · CPC title

• G02B26/0833

  the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD (G02B26/0825 takes precedence; micromechanical devices in general B81B) · CPC title

• G01S17/89

  for mapping or imaging · CPC title