Image processing device, image processing method, and program

Patent metadata

Field	Value
Publication number	US-9268410-B2
Application number	US-201214358459-A
Country	US
Kind code	B2
Filing date	Nov 29, 2012
Priority date	Feb 10, 2012
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Provided is an image processing device including a recognition unit configured to recognize an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image, a calculation unit configured to calculate an inverse matrix of the environment recognition matrix, and a manipulation control unit configured to control manipulation of a virtual object arranged in the environment in a three-dimensional manipulation amount according to a difference between a first position or a first attitude based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.

First claim

The invention claimed is:

1. An image processing device comprising:
circuitry configured to:
recognize an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image;
calculate an inverse matrix of the environment recognition matrix; and
control manipulation of a virtual object arranged in the environment appearing in the image captured by the terminal in a three-dimensional manipulation amount according to a difference between a first position or a first attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.

2. The image processing device according to claim 1 , wherein the manipulation amount is a movement amount according to the difference between the first position and the second position.

3. The image processing device according to claim 1 , wherein the manipulation amount is a rotation amount according to the difference between the first attitude and the second attitude.

4. The image processing device according to claim 1 ,
wherein the first time point is a time point at which a first user input is detected in the terminal, and
wherein the second time point is a time point at which a second user input is detected in the terminal.

5. The image processing device according to claim 4 , wherein the first user input and the second user input correspond to a start and an end of a series of manipulations, respectively.

6. The image processing device according to claim 4 , wherein the first position or the second position is a position offset in a direction on a screen of the terminal according to a user input position on the screen.

7. The image processing device according to claim 4 , wherein the circuitry is configured to specify the virtual object to be manipulated based on the first user input.

8. The image processing device according to claim 7 , wherein the circuitry is configured to specify a virtual object associated with a real object in the environment specified through the first user input as the virtual object to be manipulated.

9. The image processing device according to claim 1 , wherein at least one of functions performed by the circuitry is realized by a device present in a cloud computing environment instead of the image processing device.

10. An image processing method comprising:
recognizing an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image;
calculating an inverse matrix of the environment recognition matrix; and
controlling, using circuitry, manipulation of a virtual object arranged in the environment appearing in the image captured by the terminal in a three-dimensional manipulation amount according to a difference between a first position or a first attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.

11. A non-transitory computer readable medium including executable instructions, which when executed by a computer cause the computer to:
recognize an environment recognition matrix representing a position and an attitude of an environment appearing in an image with respect to a position and an attitude of a terminal that has captured the image;
calculate an inverse matrix of the environment recognition matrix; and
control manipulation of a virtual object arranged in the environment appearing in the image captured by the terminal in a three-dimensional manipulation amount according to a difference between a first position or a first attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a first time point and a second position or a second attitude of the terminal that has captured the image based on the inverse matrix of the environment recognition matrix recognized at a succeeding second time point.

Assignees

• Sony Corp

Inventors

• Kasahara Shunichi

Classifications

• G06F1/1694 Primary

  the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title