Automatic projection image correction system, automatic projection image correction method, and non-transitory storage medium

Patent metadata

Field	Value
Publication number	US-9344695-B2
Application number	US-201214344806-A
Country	US
Kind code	B2
Filing date	Sep 11, 2012
Priority date	Sep 15, 2011
Publication date	May 17, 2016
Grant date	May 17, 2016

Abstract

An automatic projection image correction system ( 1 ) includes: a projection image frame detection unit ( 101 ) that detects straight lines forming a frame of a screen-projected image which is projected onto a screen by a projector from a captured image obtained by capturing the screen-projected image using a camera; and a projector plane parameter estimation unit ( 102 ) that sets coordinate axes to a projector image, which is an original image of the screen-projected image, and the captured image and estimates at least one of a distance between the screen and the projector, an inclination of the screen with reference to an optical axis of the projector, and a zoom ratio of the projector, using an inclination of the optical axes of the projector and the camera, a distance between the optical centers of the projector and the camera, an intrinsic parameter of the camera, coordinate data of the detected straight lines at coordinates of the captured image, and coordinate data of straight lines corresponding to the frame at coordinates of the projector image.

First claim

The invention claimed is:

1. An automatic projection image correction system comprising:
a projection image frame detection unit that detects straight lines forming a frame of a screen-projected image which is projected onto a screen by a projector from a captured image obtained by capturing the screen-projected image using a camera; and
a projector plane parameter estimation unit that sets coordinate axes to a projector image, which is an original image of the screen-projected image, and the captured image and estimates at least one of a distance between the screen and the projector, an inclination of the screen with reference to an optical axis of the projector, and a zoom ratio of the projector, using an inclination of the optical axes of the projector and the camera, a distance between the optical centers of the projector and the camera, an intrinsic parameter of the camera, coordinate data of the detected straight lines at coordinates of the captured image, and coordinate data of straight lines corresponding to the frame at coordinates of the projector image.

2. The automatic projection image correction system according to claim 1 ,
wherein the projector plane parameter estimation unit has a conversion rule for converting a point on the coordinates of the captured image into a point on the coordinates of the projector image, using the inclination of the optical axes of the projector and the camera, the distance between the optical centers of the projector and the camera, the intrinsic parameter of the camera, the distance between the screen and the projector, the inclination of the screen with respect to the optical axis of the projector, and the zoom ratio of the projector, and
the projector plane parameter estimation unit converts the coordinate data of the detected straight lines at the coordinates of the captured image into coordinate data at the coordinates of the projector image, using the values of the inclination of the optical axes of the projector and the camera, the distance between the optical centers of the projector and the camera, and the intrinsic parameter of the camera, which are fixed values, and the values of the distance between the screen and the projector, the inclination of the screen with respect to the optical axis of the projector, and the zoom ratio of the projector, which are variable value, and estimates a plurality of the variable values such that the converted coordinate data is approximate to the coordinate data of the straight lines corresponding to the frame at the coordinates of the projector image.

3. The automatic projection image correction system according to claim 1 ,
wherein the projector plane parameter estimation unit has a conversion rule for converting a point on the coordinates of the projector image into a point on the coordinates of the captured image, using the inclination of the optical axes of the projector and the camera, the distance between the optical centers of the projector and the camera, the intrinsic parameter of the camera, the distance between the screen and the projector, the inclination of the screen with respect to the optical axis of the projector, and the zoom ratio of the projector, and
the projector plane parameter estimation unit converts the coordinate data of the straight lines corresponding to the frame at the coordinates of the projector image into coordinate data at the coordinates of the captured image, using the values of the inclination of the optical axes of the projector and the camera, the distance between the optical centers of the projector and the camera, and the intrinsic parameter of the camera, which are fixed values, and the values of the distance between the screen and the projector, the inclination of the screen with respect to the optical axis of the projector, and the zoom ratio of the projector, which are variable value, and estimates a plurality of the variable values such that the converted coordinate data is approximate to the coordinate data of the detected straight lines at the coordinates of the captured image.

4. The automatic projection image correction system according to claim 1 , further comprising:
a projection pattern generation unit that sets coordinate axes of a virtual screen, using the values of the distance between the screen and the projector and the inclination of the screen with respect to the optical axis of the projector which are estimated by the projector plane parameter estimation unit, calculates projection positions of a plurality of arbitrary points on the projector image onto the virtual screen, further using the value of the zoom ratio of the projector estimated by the projector plane parameter estimation unit, and generates the projector image which is corrected such that the screen-projected image is not distorted, using the calculation result.

5. The automatic projection image correction system according to claim 1 ,
wherein the projection image frame detection unit detects straight lines from the screen-projected image included in the captured image and detects the straight lines forming the frame of the screen-projected image from the detected straight lines.

6. The automatic projection image correction system according to according to claim 5 ,
wherein the projection image frame detection unit detects the straight lines forming the frame of the screen-projected image, based on at least one of reliability of the straight lines detected from the screen-projected image, the inclination of the straight lines, and the position of the straight lines.

7. The automatic projection image correction system according to claim 1 , further comprising:
an image composite unit that generates a composite image in which a frame of a predetermined color is superimposed on the projector image,
wherein the projection image frame detection unit detects the straight lines forming the frame of the screen-projected image from a captured image obtained by capturing the screen-projected image in the composite image using the camera.

8. The automatic projection image correction system according to claim 1 , further comprising:
a projection image frame storage unit that stores the position of the frame detected by the projection image frame detection unit; and
a camera parameter estimation unit that sets coordinate axes to the projector image, which is the original image of the screen-projected image, and the captured image and estimates the inclination of the optical axes of the projector and the camera and the distance between the optical centers of the projector and the camera, using the intrinsic parameter of the camera and the position of the frame stored in the projection image frame storage unit.

9. A non-transitory storage medium storing a program that causes a computer to function as:
a projection image frame detection unit that detects straight lines forming a frame of a screen-projected image which is projected onto a screen by a projector from a captured image obtained by capturing the screen-projected image using a camera; and
a projector plane parameter estimation unit that sets coordinate axes to a projector image, which is an original image of the screen-projected image, and the captured image and estimates at least one of a distance between the screen and the projector, an inclination of the screen with reference to an optical axis of the projector, and a zoom ratio of the projector, using an inclination of the optical axes of the projector and the camera, a distance between the optical centers of the projector and the camera, an intrinsic parameter of the camera, coordinate data of the detected straight lines at coordinates of the captured image, and coordinate data of straight lines corresponding to the frame a

Assignees

• Shibata Takashi
• Iketani Akihiko
• Senda Shuji
• Nec Corp

Inventors

• Shibata Takashi
• Iketani Akihiko
• Senda Shuji

Classifications

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• H04N9/3194

  including sensor feedback · CPC title

• H04N5/2628

  Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation · CPC title

• G09G2380/00

  Specific applications · CPC title

• H04N9/3197

  using light modulating optical valves · CPC title