Collaborative sighting

Patent metadata

Field	Value
Publication number	US-11006104-B2
Application number	US-201916688471-A
Country	US
Kind code	B2
Filing date	Nov 19, 2019
Priority date	Apr 30, 2012
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

A method includes generating calibration data by geometrically calibrating first image data from a first camera unit relative to second image data from a second camera unit based on first descriptor data and second descriptor data. The first descriptor data is based on the first image data. The second descriptor data is based on the second image data. The calibration data is generated based on first position data corresponding to the first camera unit and second position data corresponding to the second camera unit. The method includes identifying, based on the calibration data, a target location relative to the first image data. The method further includes generating an output image that includes the first image data and an indication of where the target location is relative to a scene depicted in the first image data.

First claim

What is claimed is:

1. A method comprising:
generating, at a processor, calibration data by geometrically calibrating first image data from a first camera unit relative to second image data from a second camera unit based on first descriptor data and second descriptor data, wherein the first descriptor data is based on the first image data, wherein the second descriptor data is based on the second image data, and wherein the calibration data is generated based on first position data corresponding to the first camera unit and second position data corresponding to the second camera unit, wherein geometrically calibrating includes:
identifying a plurality of descriptor pairs from the first descriptor data and the second descriptor data, wherein each descriptor pair of the plurality of descriptor pairs includes a first respective descriptor of the first descriptor data and a second respective descriptor of the second descriptor data; and
removing a subset of descriptor pairs from the plurality of descriptor pairs to generate an updated plurality of descriptor pairs, wherein the updated plurality of descriptor pairs identifies descriptor pairs that are geometrically consistent, and wherein the calibration data is based on the updated plurality of descriptor pairs;

identifying, based on the calibration data, a target location relative to the first image data; and
generating an output image, wherein the output image includes the first image data and an indication of where the target location is relative to a scene depicted in the first image data, and wherein the target location is not depicted in the scene.

2. The method of claim 1 , wherein the calibration data is generated in real time, wherein the first camera unit is coupled to a vehicle, and wherein the first camera unit is determined to be focused on a second target area in response to the first camera unit focusing on the second target area for a particular period of time.

3. The method of claim 1 , further comprising receiving the second image data from the second camera unit via a communication network.

4. The method of claim 1 , further comprising generating a homography, wherein the subset of descriptor pairs is determined based on the homography.

5. The method of claim 1 , wherein the target location corresponds to a center of a second scene depicted in the second image data.

6. The method of claim 1 , further comprising transmitting the output image to a display in real time.

7. The method of claim 1 , further comprising:
detecting a plurality of features within the first image data;
generating, by the processor, descriptors for each feature of the plurality of features, wherein the first descriptor data includes the descriptors; and
receiving the second descriptor data, wherein the second descriptor data is generated by a second processor coupled to the second camera unit, and wherein the second descriptor data is received from the second processor via a communications network.

8. The method of claim 7 , wherein the descriptors include histograms of image gradients in different directions across respective positions of the plurality of features.

9. The method of claim 1 , wherein the indicator points in a direction of the target location.

10. The method of claim 1 , further comprising:
determining that the first camera unit is focused on a second target area; and
in response to determining that the first camera unit is focused on the second target area:
generating third descriptor data based on third image data from the first camera unit; and
transmitting the third descriptor data to a second processor coupled to the second camera unit.

11. The method of claim 10 , wherein the first camera unit is determined to be focused on a second target area in response to receiving user input designating the second target area, and further comprising, in response to determining that the first camera unit focused on the second target area, transmitting target data indicating the second target area to the second camera unit.

12. The method of claim 10 , wherein the first camera unit is determined to be focused on a second target area in response to the first camera unit focusing on the second target area for a particular period of time.

13. An apparatus comprising:
a first camera unit configured to generate first image data;
a processor coupled to the first camera unit;
a memory storing instruction that, when executed by the processor, cause the processor to perform operations including:
generating calibration data by geometrically calibrating the first image data from the first camera unit relative to second image data from a second camera unit based on first descriptor data and second descriptor data, wherein the first descriptor data is based on the first image data, wherein the second descriptor data is based on the second image data, and wherein the calibration data is generated based on first position data corresponding to the first camera unit and second position data corresponding to the second camera unit, wherein geometrically calibrating includes:
identifying a plurality of descriptor pairs from the first descriptor data and the second descriptor data, wherein each descriptor pair of the plurality of descriptor pairs includes a first respective descriptor of the first descriptor data and a second respective descriptor of the second descriptor data; and
removing a subset of descriptor pairs from the plurality of descriptor pairs to generate an updated plurality of descriptor pairs, wherein the updated plurality of descriptor pairs identifies descriptor pairs that are geometrically consistent, and wherein the calibration data is based on the updated plurality of descriptor pairs;

identifying, based on the calibration data, a target location relative to the first image data; and
generating an output image, wherein the output image includes the first image data and an indication of where the target location is relative to a scene depicted in the first image data, and wherein the target location is not depicted in the scene.

14. The apparatus of claim 13 , wherein the first camera unit is coupled to a vehicle.

15. The apparatus of claim 13 , further comprising a display unit, wherein the display unit is configured to display the output image.

16. The apparatus of claim 15 , further comprising a housing, wherein the first camera unit is coupled to a first side of the housing, and wherein the display unit is coupled to a second side of the housing opposite of the first side.

17. The apparatus of claim 16 , wherein the housing is coupled to a weapon, and wherein the first camera unit and the display unit are configured to act as a sight for the weapon.

18. The apparatus of claim 13 , further comprising an Attitude and Reference System sensor, a Global Positioning System sensor, or a combination thereof.

19. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
generating calibration data by geometrically calibrating first image data from a first camera unit relative to second image data from a second camera unit based on first descriptor data and second descriptor data, wherein the first descriptor data is based on first image data, and wherein the second descriptor data is based on second image data, wherein geometrically calibrating includes:
identifying a plurality of descriptor pairs from the first descriptor data and the second descriptor data, wherein each descriptor

Assignees

• 2D3 Ltd
• Boeing Co

Inventors

• Sommerlade Eric
• Spier Emmet
• Morris Julian

Classifications

• F41G3/02 Primary

  using an independent line of sight · CPC title

• H04N17/002 Primary

  for television cameras · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• G06V10/42

  Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title