Image processing apparatus, information processing method, and program

Patent metadata

Field	Value
Publication number	US-9665168-B2
Application number	US-201514700414-A
Country	US
Kind code	B2
Filing date	Apr 30, 2015
Priority date	May 2, 2014
Publication date	May 30, 2017
Grant date	May 30, 2017

Abstract

A user's operability in an image processing apparatus such as a camera scanner is improved. The image processing apparatus has: a pickup image obtaining unit for obtaining a pickup image on a stage through an imaging unit; a distance image obtaining unit for obtaining a distance image on the stage through a solid measuring unit; a solid information obtaining unit for obtaining solid information of a solid object put on the stage based on the obtained pickup image of the solid object and the obtained distance image of the solid object; a projecting unit for projecting a projection image showing a contents area onto the solid object through a projector based on the obtained solid information; and a processing unit for executing the contents in accordance with a user's gesture to the projected projection image.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a pickup image obtaining unit configured to obtain a pickup image on a stage through an imaging unit;
a distance image obtaining unit configured to obtain a distance image on the stage through a solid measuring unit;
a solid information obtaining unit configured to obtain solid information of a solid object put on the stage on a basis of the pickup image of the solid object obtained by the pickup image obtaining unit and the distance image of the solid object obtained by the distance image obtaining unit;
a projecting unit configured to project a projection image showing a contents area onto the solid object through a projector on a basis of the solid information obtained by the solid information obtaining unit; and
a playing unit configured to play a content corresponding to the contents area in accordance with a gesture of a user to the projection image projected by the projecting unit.

2. An apparatus according to claim 1 , further comprising:
a correcting unit configured to correct a distortion of the pickup image of the solid object; and
a corresponding unit configured to make the solid information correspond to content registered in association with an image corresponding to the pickup image corrected by the correcting unit on the basis of the corrected pickup image,
and wherein the projecting unit projects the projection image to a corresponding area of the solid object in accordance with a result of the correspondence by the corresponding unit.

3. An apparatus according to claim 2 , wherein the correcting unit corrects a distortion of the pickup image of the solid object on the basis of the solid information.

4. An apparatus according to claim 2 , wherein when a distortion value of the solid object is smaller than a predetermined value, the correcting unit approximates an object area extracted from the pickup image into a square and projective-transforms into a rectangle, thereby correcting the distortion of the pickup image of the solid object.

5. An apparatus according to claim 4 , wherein even if the distortion value of the solid object is smaller than the predetermined value, when a size of the contents area corresponding to the content is smaller than a predetermined value, the correcting unit corrects the distortion of the pickup image of the solid object on the basis of the solid information.

6. An apparatus according to claim 4 , wherein even if the distortion value of the solid object is smaller than the predetermined value, when the recognized touch position by the user is closer to a boundary line of the contents area corresponding to the content by a distance larger than a predetermined value, the correcting unit corrects the distortion of the pickup image of the solid object on the basis of the solid information.

7. An apparatus according to claim 1 , further comprising:
a recognizing unit configured to recognize the gesture of the user on the stage on the basis of the distance image obtained by the distance image obtaining unit,
and wherein the playing unit plays the content in accordance with the gesture of the user to the projection image recognized by the recognizing unit.

8. An apparatus according to claim 7 , wherein when an end of a touch gesture to the solid object by the user is recognized by the recognizing unit, the solid information obtaining unit obtains the solid information again.

9. An apparatus according to claim 8 , wherein when a hand of the user is separated by a distance which is equal to or larger than a predetermined height after the touch to the solid object, the recognizing unit recognizes that the touch gesture to the solid object by the user was ended.

10. An apparatus according to claim 8 , wherein when a hand of the user is moved to a position out of a predetermined area after the touch to the solid object, the recognizing unit recognizes that the touch gesture to the solid object by the user was ended.

11. An apparatus according to claim 7 , wherein after the end of the touch gesture to the solid object by the user was recognized by the recognizing unit, when it is detected that the solid object does not exist in a predetermined area, the projecting unit projects an error display for promoting the user to return the solid object into the predetermined area.

12. An apparatus according to claim 11 , wherein when it is detected that the solid object was returned into the predetermined area in accordance with the error display projected by the projecting unit, the solid information obtaining unit obtains the solid information again.

13. An apparatus according to claim 7 , wherein when a touch gesture to the solid object by the user is recognized by the recognizing unit, the playing unit plays the content.

14. An apparatus according to claim 7 , wherein when an end of a touch gesture to the solid object by the user is recognized by the recognizing unit, the processing unit executes the contents area.

15. An apparatus according to claim 1 , wherein the projecting unit projects the projection image in an emphasis-display manner.

16. An apparatus according to claim 1 , wherein the image processing apparatus is a camera scanner.

17. An information processing method which is executed by an image processing apparatus, comprising:
a pickup image obtaining step of obtaining a pickup image on a stage through an imaging unit;
a distance image obtaining step of obtaining a distance image on the stage through a solid measuring unit;
a solid information obtaining step of obtaining solid information of a solid object put on the stage on a basis of the pickup image of the solid object obtained by the pickup image obtaining step and the distance image of the solid object obtained by the distance image obtaining step;
a projecting step of projecting a projection image showing a contents area onto the solid object through a projector on a basis of the solid information obtained by the solid information obtaining step; and
a playing step of playing a content corresponding to the contents area in accordance with a gesture of a user to the projection image projected by the projecting step.

18. A non-transitory computer-readable storage medium which stores a program for allowing a computer to execute:
a pickup image obtaining step of obtaining a pickup image on a stage through an imaging unit;
a distance image obtaining step of obtaining a distance image on the stage through a solid measuring unit;
a solid information obtaining step of obtaining solid information of a solid object put on the stage on a basis of the pickup image of the solid object obtained by the pickup image obtaining step and the distance image of the solid object obtained by the distance image obtaining step;
a projecting step of projecting a projection image showing a contents area onto the solid object through a projector on a basis of the solid information obtained by the solid information obtaining step; and
a playing step of playing a content corresponding to the contents area in accordance with a gesture of a user to the projection image projected by the projecting step.

Assignees

• Canon Kk

Inventors

• Oshima Soshi
• Takahashi Katsuyuki
• Kosaka Ryo

Classifications

• H04N23/20

  for generating image signals from infrared radiation only · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• H04N2101/00

  Still video cameras · CPC title

• G06K9/00355

  Physics · mapped topic

• G06T1/0007

  Image acquisition · CPC title