Image processing apparatus, control method and non-transitory computer-readable recording medium therefor

Patent metadata

Field	Value
Publication number	US-11416978-B2
Application number	US-202016897601-A
Country	US
Kind code	B2
Filing date	Jun 10, 2020
Priority date	Dec 25, 2017
Publication date	Aug 16, 2022
Grant date	Aug 16, 2022

Abstract

An image processing apparatus acquires a first image which captures a scene including an object from a first viewpoint position and a second image which captures a scene including the object from a second viewpoint position, and associates a coordinate position corresponding to a position of a feature of the object on the first image with a coordinate position corresponding to a position of a feature of the object on the second image. The image processing apparatus determines a partial region in the second image corresponding to a give region in the first image based on the association, generates a synthesized image by replacing an image of the given region using an image of the determined partial region, and superimposing variation data on the synthesized image.

First claim

The invention claimed is:

1. An image processing apparatus comprising:
one or more processors; and
one or more memories including instructions that, when executed by the one or more processors, cause the image processing apparatus to perform a method comprising:
(1) acquiring a first image which captures a scene including an object from a first viewpoint position and a second image which captures a scene including the object from a second viewpoint position different from the first viewpoint position;
(2) associating a coordinate position corresponding to a position of a feature of the object on the first image with a coordinate position corresponding to a position of a feature of the object on the second image;
(3) determining a partial region in the second image corresponding to a given region in the first image based on the association;
(4) generating a synthesized image by replacing an image of the given region in the first image using an image of the determined partial region;
(5) acquiring a design drawing of the object;
(6) associating the coordinate position corresponding to the position of the feature of the object on the first image with a coordinate position corresponding to a position of the feature of the object on the design drawing; and
(7) associating the coordinate position corresponding to the position of the feature of the object on the second image with the coordinate position corresponding to the position of the feature of the object on the design drawing.

2. The image processing apparatus according to claim 1 , wherein the method further comprises designating the given region in the first image.

3. The image processing apparatus according to claim 2 , wherein the method further comprises accepting a designation of a region of an arbitrary shape in the first image from a user, and
wherein the image processing apparatus designates the region of the arbitrary shape as the given region.

4. The image processing apparatus according to claim 3 , wherein the image processing apparatus is configured to be capable of accepting a plurality of regions of an arbitrary shape, and
wherein the image processing apparatus designates, as the given region, a region obtained by a logical sum or a logical product of the plurality of regions of the arbitrary shape.

5. The image processing apparatus according to claim 2 , wherein the method further comprises accepting a designation of an image feature from a user, and
wherein the image processing apparatus designates a region in the first image similar to the image feature as the given region.

6. The image processing apparatus according to claim 1 , wherein the method further comprises arranging and displaying the first image and the second image on a display unit, and
wherein the image processing apparatus performs control so as to display, synchronously with the given region, the determined partial region in the second image.

7. The image processing apparatus according to claim 1 , wherein the method further comprises superimposing and displaying the second image having undergone translucent processing on the first image in a display unit, and
wherein the image processing apparatus performs control so as to superimpose and display the second image having undergone the translucent processing on the first image based on the association.

8. The image processing apparatus according to claim 7 , wherein the method further comprises designating a transparency in the translucent processing.

9. The image processing apparatus according to claim 1 , wherein the method further comprises performing image processing for correcting a shift occurring between the partial region and the first image in the synthesized image.

10. The image processing apparatus according to claim 9 , wherein the image processing apparatus corrects a positional shift occurring between the partial region and the first image by deforming the image of the partial region based on an association relationship between coordinates specified from a boundary of the given region and coordinates specified from a boundary of the partial region.

11. The image processing apparatus according to claim 10 , wherein the variation that has occurred in the object is a crack that has occurred in a structure, and
wherein the image processing apparatus (a) specifies an end point of a crack detected from the first image on the boundary of the given region and an endpoint of a crack detected from the second image on the boundary of the partial region, and (b) deforms the partial region based on association of the end points falling within a predetermined range among the specified end points.

12. The image processing apparatus according to claim 9 , wherein the image processing apparatus corrects a color shift occurring between the partial region and the first image by adjusting any or all of a hue, a saturation, and a brightness of the partial region based on any or all of a hue, a saturation, and a brightness of the first image.

13. The image processing apparatus according to claim 9 , wherein the image processing apparatus corrects a color shift occurring between the partial region and the first image by adjusting any or all of a hue, a saturation, and a brightness of the first image based on any or all of a hue, a saturation, and a brightness of the partial region.

14. The image processing apparatus according to claim 9 , wherein
the image processing apparatus generates the synthesized image by superimposing, on the given region in the first image, the image of the partial region whose transparency is set, and
wherein the image processing apparatus corrects a color shift occurring between the partial region and the first image by setting the transparency that continuously increases from a center to an outside in the partial region.

15. The image processing apparatus according to claim 14 , wherein
the image processing apparatus generates the synthesized image by superimposing an image of an extended partial region larger than the determined partial region on the given region in the first image, and
wherein the image processing apparatus corrects a color shift occurring between the partial region and the first image by setting, in a region of the extended partial region extending from the partial region, a transparency that continuously increases from a side in contact with the partial region to an outside of the extended partial region.

16. The image processing apparatus according to claim 1 , wherein the method further comprises deriving, based on the association, a coordinate conversion parameter that mutually converts a coordinate position in the object included in the first image and a coordinate position in the object included in the second image, and
wherein the image processing apparatus determines, based on the coordinate conversion parameter, the partial region in the second image corresponding to the given region in the first image.

17. The image processing apparatus according to claim 1 , wherein the method further comprises displaying variation data representing a variation that has occurred in the object by superimposing the variation data on the generated synthesized image.

18. The image processing apparatus according to claim 17 , wherein the method further comprises detecting a variation that has occurred in the object based on analysis of the synthesized image, and
wherein, in the displaying, the image processing apparatus displays the variation data representing the detected variation by superimposing the variation data on the

Assignees

• Canon Kk

Inventors

• Iwabuchi Kohei
• Anabuki Mahoro
• Kuwabara Nobuaki

Classifications

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T7/0006 Primary

  using a design-rule based approach · CPC title

• G06T1/0007

  Image acquisition · CPC title

• G06T2207/30132

  Masonry; Concrete · CPC title