Information processing apparatus and information processing method

Patent metadata

Field	Value
Publication number	US-12003899-B2
Application number	US-202118001119-A
Country	US
Kind code	B2
Filing date	May 7, 2021
Priority date	Jun 17, 2020
Publication date	Jun 4, 2024
Grant date	Jun 4, 2024

Abstract

Provided is an information processing apparatus that includes a transformation matrix calculation section, a scale calculation section, and an input display generation section. The transformation matrix calculation section calculates a transformation matrix that converts the coordinates of an input position of an input device within a projection region projected from a display apparatus from the coordinates of an image coordinate system of a sensor apparatus having detected the input position to the coordinates of a screen coordinate system of the display apparatus. The scale calculation section calculates a scale correction value for a display mode of the locus of the input position in reference to distance information regarding the distance between the display apparatus and the input position. The input display generation section generates an input image depicting the locus of the input position through the use of the transformation matrix and the scale correction value.

First claim

The invention claimed is:

1. An information processing apparatus, comprising:
a transformation matrix calculation section configured to calculate a transformation matrix for converting coordinates of an input position of an input device within a projection region projected from a display apparatus from coordinates of an image coordinate system of a sensor apparatus having detected the input position to coordinates of a screen coordinate system of the display apparatus;
a scale calculation section configured to calculate a scale correction value for a display mode of a locus of the input position in reference to distance information regarding a distance between the display apparatus and the input position; and
an input display generation section configured to generate an input image depicting the locus of the input position through use of the transformation matrix and the scale correction value.

2. The information processing apparatus according to claim 1 , further comprising:
a content display generation section configured to generate a content image to be displayed in the projection region together with the input image by the display apparatus; and
a control section configured to combine the input image with the content image.

3. The information processing apparatus according to claim 1 , wherein the scale correction value is a correction value for thickness, lightness, or saturation of the locus or for contrast of a contour.

4. The information processing apparatus according to claim 3 , wherein a lower-limit value of the thickness is preset according to the distance.

5. The information processing apparatus according to claim 1 , further comprising a reference point display generation section configured to generate a reference point image depicting at least four reference points displayed in the projection region by the display apparatus, wherein
the input device is a pen device that displays the input position with infrared light,
the sensor apparatus is an infrared stereo camera that is mounted on the display apparatus to detect the input position and the distance,
the transformation matrix calculation section is further configured to calculate the transformation matrix according to the reference point image and an infrared image that is detected by the infrared stereo camera and depicted by the infrared light whose reference points are outputted from the pen device, and
the scale calculation section is further configured to calculate the scale correction value for each of the reference points.

6. The information processing apparatus according to claim 5 , wherein the scale calculation section is further configured to:
perform linear interpolation to calculate a distance from the display apparatus to a pixel between the reference points within the reference point image; and
calculate the scale correction value for the pixel according to the calculated distance.

7. The information processing apparatus according to claim 5 , further comprising an input coordinate detection section configured to:
extract a drawing region of the infrared light from the infrared image; and
detect a centroid of the drawing region as position coordinates of the reference points.

8. The information processing apparatus according to claim 1 , wherein, after each change in a projection direction of the display apparatus,
the transformation matrix calculation section is further configured to
calculate the transformation matrix, and the scale calculation section is further configured to calculate the scale correction value.

9. An information processing method, comprising:
calculating a transformation matrix that converts coordinates of an input position of an input device within a projection region projected from a display apparatus from coordinates of an image coordinate system of a sensor apparatus having detected the input position to coordinates of a screen coordinate system of the display apparatus;
calculating a scale correction value for a display mode of a locus of the input position in reference to distance information regarding a distance between the display apparatus and the input position; and
generating an input image depicting the locus of the input position through use of the transformation matrix and the scale correction value.

10. A non-transitory computer-readable medium having stored thereon, computer-readable instructions which, when executed by a computer communicatively connected to each of an input device, a display apparatus, and a sensor apparatus, cause the computer to execute operations, the operations comprising:
calculating a transformation matrix that converts coordinates of an input position of the input device within a projection region projected from the display apparatus from coordinates of an image coordinate system of the sensor apparatus having detected the input position to coordinates of a screen coordinate system of the display apparatus;
calculating a scale correction value for a display mode of a locus of the input position in reference to distance information regarding a distance between the display apparatus and the input position; and
generating an input image depicting the locus of the input position through use of the transformation matrix and the scale correction value.

Assignees

• Sony Group Corp

Inventors

• Aoki Yu
• Ikeda Takuya
• Ozaki Honoka

Classifications

• H04N9/3188 Primary

  Scale or resolution adjustment (scaling in general G06T3/40; resolution modifying circuits for displays in general G09G5/391) · CPC title

• G06F3/03542

  Light pens for emitting or receiving light · CPC title

• G06T3/4007

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/10012

  Stereo images · CPC title