Information processing device, information processing method, and program for recognizing attitude of a plane

Patent metadata

Field	Value
Publication number	US-9355451-B2
Application number	US-201214238978-A
Country	US
Kind code	B2
Filing date	Aug 14, 2012
Priority date	Aug 24, 2011
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

According to an embodiment of the present disclosure, there is provided an information processing device including a data acquisition unit configured to acquire sensor data indicating the direction of gravity exerted on an imaging device configured to image an image in which a physical space is projected, a decision unit configured to decide a relative attitude of a plane in the physical space with respect to the image based on the sensor data, and a conversion unit configured to perform conversion between a three-dimensional position of a given point on the plane and a two-dimensional position in the corresponding image using the attitude decided by the decision unit.

First claim

The invention claimed is:

1. An information processing device comprising:
a data acquisition unit configured to acquire sensor data indicating the direction of gravity exerted on an imaging device configured to image an image in which a physical space is projected;
a decision unit configured to decide a relative attitude of a plane in the physical space with respect to the image based on the sensor data; and
a conversion unit configured to perform conversion between a three-dimensional position of a given point on the plane and a two-dimensional position in the image using the attitude decided by the decision unit,
wherein in an event the plane is a vertical plane, the decision unit is configured to decide an attitude of the vertical plane based on the sensor data indicating the direction of gravity and map data indicating the positional relationship between the imaging device and the vertical plane.

2. The information processing device according to claim 1 , wherein the given point on the plane is a feature point selected at a time of initialization of a simultaneous localization and mapping (SLAM) method.

3. The information processing device according to claim 2 , further comprising:
a recognition unit configured to recognize an object projected in the image,
wherein the decision unit is further configured to decide a position of the plane in the physical space using known data of a real size of the object recognized by the recognition unit.

4. The information processing device according to claim 1 , wherein the given point on the plane is a point constituting an annotation for augmented reality.

5. The information processing device according to claim 3 ,
wherein the decision unit is configured to decide a hypothetical plane having the attitude decided based on the sensor data and a tentative position,
wherein the information processing device further includes a display control unit configured to superimpose the annotation on the image, and
wherein the annotation is superimposed at a two-dimensional position in the image converted by the conversion unit from a three-dimensional position on the hypothetical plane.

6. The information processing device according to claim 4 , wherein the decision unit is further configured to decide a position of the plane in which the annotation is to be disposed using a display size of the annotation to be displayed.

7. The information processing device according to claim 6 , further comprising:
a user interface unit configured to enable a user to designate the display size.

8. The information processing device according to claim 1 , wherein the decision unit is further configured to decide a position of the plane using data relating to the distance between the plane in the physical space which is projected in the image and the imaging device.

9. The information processing device according to claim 1 ,
wherein in an event the plane is a horizontal plane,
the decision unit is configured to determine whether the imaging device faces an upper direction or a lower direction based on the sensor data, and decide the attitude of the horizontal plane according to a result of the determination.

10. The information processing device according to claim 1 ,
wherein in an event the plane is a vertical plane,
the decision unit is configured to, alternatively, decide the attitude of the vertical plane based on the sensor data indicating the direction of gravity and a restriction of a pre-defined direction.

11. The information processing device according to claim 1 , wherein the decision unit is configured to estimate a vertical plane projected in the image using sensor data indicating a direction of geomagnetism from one or more vertical plane candidates included in the map data, and decide an attitude of the estimated vertical plane based on the map data and the direction of gravity.

12. The information processing device according to claim 1 , wherein the decision unit is configured to decide an attitude of the vertical plane designated by a user from one or more vertical plane candidates included in the map data based on the map data and the direction of gravity.

13. The information processing device according to claim 1 , further comprising:
a user interface unit configured to dispose a user interface that enables a user to designate an attitude of the plane in an augmented reality space with reference to an attitude of a horizontal plane or the attitude of the vertical plane decided based on the sensor data.

14. The information processing device according to claim 13 , wherein the disposition of the user interface is maintained in the augmented reality space over a plurality of frames.

15. The information processing device according to claim 14 ,
wherein the user interface has an operation plane configured to be three-dimensionally rotation-operable, and
wherein the decision unit is configured to decide an attitude of the plane using an attitude of the operation plane operated by the user so as to be in parallel with the plane.

16. The information processing device according to claim 1 , wherein the information processing device is a terminal device configured to further have the imaging device and a sensor configured to output the sensor data.

17. The information processing device according to claim 1 , wherein at least one of the data acquisition unit, the decision unit, and the conversion unit is installed in a device present in a cloud computing environment instead of in the information processing device.

18. An information processing method of a computer configured to control an information processing device, the method comprising:
acquiring sensor data indicating the direction of gravity exerted on an imaging device configured to image an image in which a physical space is projected;
deciding a relative attitude of a plane in the physical space with respect to the image based on the sensor data; and
performing conversion between a three-dimensional position of a given point on the plane and a two-dimensional position in the image using the decided attitude,
wherein in an event of a vertical plane, an attitude of the vertical plane is decided based on the sensor data indicating the direction of gravity and map data indicating the positional relationship between the imaging device and the vertical plane.

19. A non-transitory computer-readable storage medium having stored thereon, a set of computer-executable instructions for causing a computer to function as:
a data acquisition unit configured to acquire sensor data indicating the direction of gravity exerted on an imaging device configured to image an image in which a physical space is projected;
a decision unit configured to decide a relative attitude of a plane in the physical space with respect to the image based on the sensor data; and
a conversion unit configured to perform conversion between a three-dimensional position of a given point on the plane and a two-dimensional position in the image using the attitude decided by the decision unit,
wherein in an event of a vertical plane, the decision unit is configured to decide an attitude of the vertical plane based on the sensor data indicating the direction of gravity and map data indicating the positional relationship between the imaging device and the vertical plane.

Assignees

• Oi Kenichiro
• Ishiwatari Hiroyuki
• Sony Corp

Inventors

• Oi Kenichiro
• Ishiwatari Hiroyuki

Classifications

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G01C21/3647

  Guidance involving output of stored or live camera images or video streams · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title