Method and system for determining a relation between a first scene and a second scene

Patent metadata

Field	Value
Publication number	US-9792701-B2
Application number	US-201114355144-A
Country	US
Kind code	B2
Filing date	Nov 8, 2011
Priority date	Nov 8, 2011
Publication date	Oct 17, 2017
Grant date	Oct 17, 2017

Abstract

The present invention relates to a system ( 200 ) and method for determining a relation between a first scene and a second scene. The method comprises the steps of generating at least one sensor image of a first scene with at least one sensor; accessing information related to at least one second scene, said second scene encompassing said first scene, and matching the sensor image with the second scene to map the sensor image onto the second scene. The step of accessing information related to the at least one second scene comprises accessing a 3D map comprising geocoded 3D coordinate data. The mapping involves associating geocoding information to a plurality of positions in the sensor image based on the coordinate data of the second scene.

First claim

The invention claimed is:

1. Method for determining a relation between a first scene and a second scene, said method comprising the steps of
generating at least one sensor image of a first scene with at least one sensor;
accessing information related to at least one second scene by a processing and control module, said second scene encompassing said first scene; and
matching, by the processing and control module, the sensor image with the second scene to map the sensor image onto the second scene,
wherein:
the step of accessing information related to the at least one second scene comprises:
accessing a textured 3D map comprising geocoded 3D coordinate data by the processing and control module, wherein the textured 3D map comprises a 3D model having texture information associated to one or more surfaces of the 3D model; and
extracting, from the textured 3D map and by the processing and control module, the texture information comprising the corresponding geocoded 3D coordinate data related to the at least one second scene, said extracting generating 2D extracted texture information; and

the step of matching the sensor image with the second scene to map the sensor image onto the second scene comprises:
matching, by the processing and control module, the 2D extracted texture information corresponding to at least one of the one or more surfaces of the 3D model and comprising the corresponding geocoded 3D coordinate data with 2D image texture information of the sensor image, the 2D image texture information of the sensor image corresponding to one or more surfaces within the sensor image, and
associating, by the processing and control module, the extracted corresponding geocoded 3D coordinate data to a plurality of positions in the sensor image based on the matching of the 2D extracted texture information with the 2D image texture information of the sensor image.

2. Method according to claim 1 , wherein the 3D map comprises a 3D model of the environment comprising 3D coordinate data given in a geo-referenced coordinate system.

3. Method according to claim 1 , further comprising:
displaying the sensor image on a display;
receiving a selection of a point on the displayed image of the sensor image;
extracting coordinate information related to the selected point in the sensor image.

4. Method according to claim 1 , further comprising a step of forming a geocoded reference image from the second scene, wherein a geographical extension of the geocoded reference image corresponds to a geographical extension of the sensor image.

5. Method according to claim 4 , further comprising a step of identifying geocoded additional information from the geocoded sensor image in relation to the corresponding geocoded reference image.

6. Method according to claim 5 , wherein the geocoded additional information comprises information related to a difference in at least one of shape or size of at least one object in the sensor image in relation to the corresponding geocoded reference image.

7. Method according to claim 5 , wherein the additional information comprises information relating to at least one texture deviation in the sensor image in relation to the corresponding geocoded reference image.

8. Method according to claim 5 , wherein the additional information comprises additional geocoded coordinate data providing enhanced resolution in relation to the reference image.

9. Method according to claim 8 , further comprising a step of updating the geocoded reference image based on the additional information.

10. Method according to claim 5 , wherein the at least one sensor comprises an IR sensor, and wherein the geocoded additional information comprises IR data.

11. Method according to claim 5 , further comprising a step of presenting the geocoded additional information.

12. System for determining a relation between a first scene and a second scene, said system comprising:
at least one sensor configured to capture at least one image of the first scene;
a map comprising geographic information; and
a processing and control module comprising a processor and coupled to a storage medium, wherein the processing and control module accesses from the map information related to at least one second scene, said second scene encompassing said sensor image, and matches the sensor image with the second scene to map the sensor image onto the second scene,
wherein:
the map is a textured 3D map comprising geocoded 3D coordinate data, wherein the textured 3D map comprises a 3D model having texture information associated to one or more surfaces of the 3D model;
the accessing of the map by the processing and control module comprises:
accessing the textured 3D map, and
extracting, from the textured 3D map, the texture information comprising the corresponding geocoded 3D coordinate data related to the at least one second scene, said extracting generating 2D extracted texture information; and

the processing and control module matches the 2D extracted texture information corresponding to at least one of the one or more surfaces of the 3D model and comprising the corresponding geocoded 3D coordinate data with 2D image texture information of the sensor image, the 2D image texture information of the sensor image corresponding to one or more surfaces within the sensor image, and associates the extracted corresponding geocoded 3D coordinate data to a plurality of positions in the sensor image based on the coordinate data of the second scene based on the matching of the 2D extracted texture information with the 2D image texture information of the sensor image.

13. System according to claim 12 , wherein the 3D map comprises a 3D model of the environment comprising 3D coordinate data given in a geo-referenced coordinate system.

14. System according to claim 12 further comprising means for extracting coordinate information related to a selected point in the sensor image.

15. System according to claim 14 , further comprising a display configured to display the sensor image and selection means for selecting the point from the displayed sensor image.

16. System according to claim 12 , wherein the processing and control module is configured to form a geocoded reference image from the second scene, wherein a geographical extension of the geocoded reference image corresponds to the geographical extension of a sensor image.

17. System according to claim 16 , wherein the processing and control module further is configured to identify geocoded additional information from the geocoded sensor image in relation to the corresponding geocoded reference image.

18. System according to claim 17 , wherein the geocoded additional information comprises information relating to at least one of:
at least one of addition or removal of at least one object in the sensor image in relation to the corresponding geocoded reference image;
at least one of a difference in at least one of shape or size of at least one object in the sensor image in relation to the corresponding geocoded reference image; or
at least one of texture deviation in the sensor image in relation to the corresponding geocoded reference image.

19. System according to claim 17 wherein the additional information comprises additional geocoded coordinate data providing enhanced accuracy in relation to the reference image.

20. System according to claim 19 , wherein the 3D map is updated with the additional information based on the geocoded reference image based on the additional information.

21

Assignees

• Olofsson Michael
• Isaksson Folke
• Brege Magnus
• Andersson Ingmar
• Carlbom Per
• Borg Johan
• Haglund Leif
• Rosenqvist Fredrik
• Saab Ab

Inventors

• Olofsson Michael
• Isaksson Folke
• Brege Magnus
• Andersson Ingmar
• Carlbom Per
• Borg Johan
• Haglund Leif
• Rosenqvist Fredrik

Classifications

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T7/75 Primary

  involving models · CPC title

• G06T2207/30181

  Earth observation · CPC title

• G01C11/06

  by comparison of two or more pictures of the same area · CPC title