Image processing device, image processing method, and computer program product

Patent metadata

Field	Value
Publication number	US-9846966-B2
Application number	US-201514611450-A
Country	US
Kind code	B2
Filing date	Feb 2, 2015
Priority date	Feb 12, 2014
Publication date	Dec 19, 2017
Grant date	Dec 19, 2017

Abstract

An image processing device includes: an augmented-reality marker detecting unit that detects, from a captured image, an augmented reality marker indicating that a virtual object is to be displayed; an operation-marker detecting unit that detects, from the captured image, an area of an operation marker that is provided so as to correspond to the augmented reality marker; an operation detecting unit that detects, from the area of the operation marker, a user's operation on the operation marker; a virtual-object control unit that deforms a virtual object corresponding to the augmented reality marker in accordance with the user's operation; and a drawing unit that draws the deformed virtual object in the captured image in a superimposed manner.

First claim

What is claimed is:

1. An image processing device comprising:
circuitry configured to
detect, from a captured image, an augmented reality marker indicating that a virtual object is to be displayed,
detect, from the captured image, an area of an operation marker that is provided so as to correspond to the augmented reality marker, and determine that an operation of a user is performed when the operation marker is touched by a part of a body of the user for a predetermined period even when the part of the body is separated from the operation marker,
deform a virtual object corresponding to the augmented reality marker based on the operation of the user, and
draw the deformed virtual object in the captured image in a superimposed manner.

2. The image processing device according to claim 1 , wherein
the operation marker is located at a predetermined position with respect to the augmented reality marker, and
the circuitry is configured to detect the area of the operation marker in the captured image based on a position of the area of the augmented reality marker.

3. The image processing device according to claim 1 , wherein
the operation marker is detectable independently from the augmented reality marker, and
the circuitry is configured to detect, from the captured image, the area of the operation marker independently from the augmented reality marker.

4. The image processing device according to claim 1 , wherein
the circuitry is configured to detect a difference between a partial image of the area of the operation marker in the captured image and a reference image of the operation marker that is previously stored and, when the difference is greater than a predetermined threshold, determine that the operation is performed on the corresponding operation marker.

5. The image processing device according to claim 1 , further comprising a display configured to display the captured image where the virtual object is superimposed by the circuitry.

6. An image processing method comprising:
detecting, from a captured image, an augmented reality marker indicating that a virtual object is to be displayed;
detecting, from the captured image, an area of an operation marker that is provided so as to correspond to the augmented reality marker;
determining that an operation of a user is performed when the operation marker is touched by a part of a body of the user for a predetermined period even when the part of the body is separated from the operation marker;
deforming a virtual object corresponding to the augmented reality marker based on the operation of the user; and
drawing the deformed virtual object in the captured image in a superimposed manner.

7. A computer program product comprising a non-transitory computer-readable medium having computer readable program codes, the program codes when executed causing the computer to perform:
detecting, from a captured image, an augmented reality marker indicating that a virtual object is to be displayed;
detecting, from the captured image, an area of an operation marker that is provided so as to correspond to the augmented reality marker;
determining that an operation of a user is performed when the operation marker is touched by a part of a body of the user for a predetermined period even when the part of the body is separated from the operation marker;
deforming a virtual object corresponding to the augmented reality marker in based on the operation of the user; and
drawing the deformed virtual object in the captured image in a superimposed manner.

8. An image processing device comprising:
circuitry configured to
detect, from a captured image, an augmented reality marker indicating that a virtual object is to be displayed,
detect, from the captured image, an area of at least one operation marker that is provided so as to correspond to the augmented reality marker, and detect, as an operation of a user, a movement of a part of a body of the user with respect to the at least one operation marker,
change a movement of the virtual object in accordance with the movement of the part of the body of the user, and
draw the changed virtual object in the captured image in a superimposed manner.

9. The image processing device according to claim 8 , wherein the circuitry is configured to detect the operation of the user when the part of the body of the user moves on the at least one operation marker.

10. An image processing method comprising:
detecting, from a captured image, an augmented reality marker indicating that a virtual object is to be displayed;
detecting, from the captured image, an area of at least one operation marker that is provided so as to correspond to the augmented reality marker;
detecting, as an operation of a user, a movement of a part of a body of the user with respect to the at least one operation marker;
changing a movement of the virtual object in accordance with the movement of the part of the body of the user; and
drawing the changed virtual object in the captured image in a superimposed manner.

11. A computer program product comprising a non-transitory computer-readable medium having computer readable program codes, the program codes when executed causing the computer to perform:
detecting, from a captured image, an augmented reality marker indicating that a virtual object is to he displayed;
detecting, from the captured image, an area of at least one operation marker that is provided so as to correspond to the augmented reality marker;
detecting, as an operation of a user, a movement of a part of a body of the user with respect to the at least one operation marker;
changing a movement of the virtual object in accordance with the movement of the part of the body of the user; and
drawing the changed virtual object in the captured image in a superimposed manner.

Assignees

• Sadasue Tamon
• Ricoh Co Ltd

Inventors

• Sadasue Tamon

Classifications

• G06F3/002

  Specific input/output arrangements not covered by G06F3/01 - G06F3/16 (other optical apparatus G02B27/00) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G09G2340/04

  Changes in size, position or resolution of an image · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/147 Primary

  using display panels · CPC title