Projection image correcting apparatus, method for correcting image to be projected, and program

Patent metadata

Field	Value
Publication number	US-9672602-B2
Application number	US-201514642021-A
Country	US
Kind code	B2
Filing date	Mar 9, 2015
Priority date	Mar 10, 2014
Publication date	Jun 6, 2017
Grant date	Jun 6, 2017

Abstract

A projection image correcting apparatus for correcting, according to a three-dimensional shape of a projection surface, an original image to be projected, includes a projecting unit configured to project a pattern image on the projection surface, by controlling an image projecting device; a capturing unit configured to obtain a captured image of the projection surface on which the pattern image is projected, by controlling an image capturing device; a feature point extracting unit configured to extract feature points in the captured image corresponding to feature points in the pattern image; and a three-dimensional coordinates measuring unit configured to measure, by triangulation using the extracted feature points, three-dimensional coordinates of a plurality of measuring points on the projection surface corresponding to the feature points; and a reliability calculating unit configured to calculate reliabilities of the measuring points.

First claim

What is claimed is:

1. A projection image correcting apparatus for correcting, according to a three-dimensional shape of a projection surface, an original image to be projected, comprising:
memory storing computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions such that the one or more processors are configured to,
project a pattern image on the projection surface, by controlling an image projecting device,
obtain a captured image of the projection surface on which the pattern image is projected, by controlling an image capturing device,
extract feature points in the captured image corresponding to feature points in the pattern image,
measure, by triangulation using the extracted feature points, three-dimensional coordinates of a plurality of measuring points on the projection surface corresponding to the feature points,
calculate reliabilities of the measuring points,
convert the three dimensional coordinates of the measuring points into two-dimensional coordinates as defined in a view from a first viewpoint and define a reliability area including the converted measuring points that respectively have the reliabilities exceeding a certain threshold,
create a reference table in which a distortion of a projection image viewed from the first viewpoint is expressed by association of the two-dimensional coordinates on the reliability area with two-dimensional coordinates on an image projecting coordinate system,
correct the original image so that a distortion for offsetting the distortion expressed in the reference table is given to the original image, and
find a projection area that is a rectangular area, having the same aspect ratio as that of the original image, included in the reliability area, and reduce the original image to be disposed within the projection area, before correcting the original image based on the reference table,
wherein the reliabilities are defined based on at least one of lens aberration and detection errors of the feature points.

2. The projection image correcting apparatus as claimed in claim 1 , wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to
correct a value of a first pixel of the original image into a value of black, in a case where two-dimensional coordinates of the reliability area corresponding to the first pixel have failed to be found in the reference table, and
retrieve a value of a second pixel of the original image whose two-dimensional coordinates on an image projecting coordinate system are associated with the two-dimensional coordinates of the reliability area corresponding to the first pixel in the reference table, thereby correcting the value of the first pixel into the retrieved value of the second pixel, in a case where the two-dimensional coordinates of the reliability area corresponding to the first pixel are found in the reference table.

3. The projection image correcting apparatus as claimed in claim 1 , wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to calculate a reliability of the reliabilities defined based on the lens aberration by formula (1),
RELIABILITY=1/ d c +1/ d p   (1)

wherein “d c ” is a distance between the feature point and the center of the axis of the light beam of the captured image, and “d p ” is a distance between the feature point and the center of the axis of the light beam of the pattern image.

4. The projection image correcting apparatus as claimed in claim 1 , wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to,
calculate a reliability of the reliabilities based on a distance between a line of projecting sight and a line of capturing sight in a case where the three-dimensional coordinates of the measuring point are measured by a midpoint method, and
calculate the reliability based on a reprojection error in a case where the three-dimensional coordinates of the measuring point are measured by a method based on Reprojection Error Minimization.

5. A method for correcting, according to a three-dimensional shape of a projection surface, an original image to be projected, the method comprising: projecting a pattern image on the projection surface, by controlling an image projecting device;
obtaining a captured image of the projection surface on which the pattern image is projected, by controlling an image capturing device;
extracting feature points in the captured image corresponding to feature points in the pattern image;
measuring, by triangulation using the extracted feature points, three-dimensional coordinates of a plurality of measuring points on the projection surface corresponding to the feature points;
calculating reliabilities of the measuring points;
converting the three dimensional coordinates of the measuring points into two-dimensional coordinates as defined in a view from a first viewpoint and defining a reliability area including the converted measuring points that respectively have the reliabilities exceeding a certain threshold;
creating a reference table in which a distortion of a projection image viewed from the first viewpoint is expressed by association of the two-dimensional coordinates on the reliability area with two-dimensional coordinates on an image projecting coordinate system;
correcting the original image so that a distortion for offsetting the distortion expressed in the reference table is given to the original image; and
finding a projection area that is a rectangular area, having the same aspect ratio as that of the original image, included in the reliability area, and reducing the original image to be disposed within the projection area, before correcting the original image based on the reference table,
wherein the reliabilities are defined based on at least one of lens aberration and detection errors of the feature points.

6. The method as claimed in claim 5 , wherein the correcting comprises:
correcting a value of a first pixel of the original image into a value of black, in a case where two-dimensional coordinates of the reliability area corresponding to the first pixel have failed to be found in the reference table, and
retrieving a value of a second pixel of the original image, whose two-dimensional coordinates on an image projecting coordinate system are associated with the two-dimensional coordinates of the reliability area corresponding to the first pixel in the reference table, thereby correcting the value of the first pixel into the retrieved value of the second pixel, in a case where the two-dimensional coordinates of the reliability area corresponding to the first pixel are found in the reference table.

7. The method as claimed in claim 5 , wherein in the calculating of the reliabilities, a reliability of the reliabilities is defined based on the lens aberration by formula (1),
RELIABILITY=1/ d c +1/ d p   (1)

wherein “d c ” is a distance between the feature point and the center of the axis of the light beam of the captured image, and “d p ” is a distance between the feature point and the center of the axis of the light beam of the pattern image.

8. The method as claimed in claim 5 , wherein the calculating the reliabilities comprises:
calculating a reliability of the reliabilities based on a distance between a line of projecting sight and a line of capturing sight in a case where the three-dimensional coordinates of the measuring point are measured by a midpoint method, and
calculating the reliability based on a reprojection erro

Assignees

• Hara Takayuki
• Ricoh Co Ltd

Inventors

• Hara Takayuki

Classifications

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G06T5/006 Primary

  Physics · mapped topic

• H04N9/3185

  Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14) · CPC title

• H04N9/3194

  including sensor feedback · CPC title

• G06T5/80 Primary

  Geometric correction · CPC title