Information processing device, information processing method, and program

Patent metadata

Field	Value
Publication number	US-10389937-B2
Application number	US-201615176577-A
Country	US
Kind code	B2
Filing date	Jun 8, 2016
Priority date	Feb 17, 2014
Publication date	Aug 20, 2019
Grant date	Aug 20, 2019

Abstract

To provide an information processing device, an information processing method, and a program capable of sharing a space while maintaining the degree of freedom of a visual line. An information processing device according to the present disclosure includes: a control unit configured to perform control in a manner that a display image generated based on image information which is generated through imaging of an imaging device mounted on a moving object moving in a space, imaging-device posture information which is information regarding a posture of the imaging device, and user view information which is obtained from a user manipulation device manipulated by a user and specifies a region that the user desires to view is displayed in a display region viewed by the user.

First claim

The invention claimed is:

1. An information processing device comprising:
circuitry configured to control a display terminal to:
acquire a captured image obtained by an imaging device via a network server;
display a partial image of the captured image in a display region of the display terminal, wherein
the partial image corresponds to a part of the captured image that is specified based on view information,
the view information is obtained from the display terminal, and
the captured image is wider than the partial image;
perform correction on the captured image so as to suppress a change in the captured image resulting from a change of a posture of the imaging device; and

display a superimposed image over the partial image when the correction on the captured image is performed,

wherein the superimposed image represents a relative coordinate system that is fixed to the imaging device.

2. The information processing device according to claim 1 ,
wherein in the correction performed on the captured image, the circuitry is further configured to:
acquire a rotation component of a change from a first captured image to a second captured image, wherein the second captured image is obtained by the imaging device after the first captured image; and
reversely rotate the second captured image, based on the rotation component, to match the first local feature amounts of the first captured image with second local feature amounts of the second captured image.

3. The information processing device according to claim 1 , wherein the superimposed image indicates the posture of the imaging device based on a signal from at least one sensor that is installed in an apparatus including the imaging device.

4. The information processing device according to claim 1 ,
wherein the superimposed image includes an object that is not moved on the relative coordinate system and indicates at least one of a numerical value or a letter.

5. The information processing device according to claim 1 ,
wherein the circuitry is further configured to control the display terminal to display the superimposed image to rotate, when viewed from a user of the display terminal, in a real space in which the imaging device is present.

6. The information processing device according to claim 5 ,
wherein the circuitry is further configured to control the display terminal to offset a viewpoint of the user from a position of the imaging device in the real space.

7. The information processing device according to claim 1 ,
wherein the circuitry is further configured to fix an annotation object to a specific position of an absolute coordinate system, which is fixed to a real space in which the imaging device is present, while suppressing the change in the captured image resulting from the change of the posture of the imaging device.

8. The information processing device according to claim 1 ,
wherein the circuitry is further configured to control the display terminal to reduce a reproduction speed of the partial image in accordance with an increase of a movement amount of the imaging device.

9. The information processing device according to claim 1 ,
wherein the circuitry is further configured to control the display terminal to change a viewpoint a user of the display terminal based on a designation from the user along a direction including a present viewpoint of the user.

10. The information processing device according to claim 1 ,
wherein the circuitry is further configured to control the display terminal to:
set, based on a manipulation performed on at least one of the imaging device and the display terminal, a visualization state of the captured image and the superimposed image to one of a first state or a second state,
wherein in the first state, the correction on the captured image is not performed and a position of the superimposed image is prevented from moving on the display region of the display terminal even when the posture of the imaging device is changed, and
wherein in the second state, the correction on the captured image is performed and the position of the superimposed image is moved on the display region of the display terminal to be rotated in a real space in which the imaging device is present when viewed from.

11. The information processing device according to claim 1 ,
wherein the is display terminal comprises a wearable device configured to be mounted on a user, and
wherein the circuitry is further configured to change the view information according to a change of an orientation of the wearable device to provide a first-person viewpoint image to the user independently from a change of the posture of the imaging device.

12. The information processing device according to claim 1 ,
wherein the imaging device is mounted onto one of a self-propelled object that propels itself in a real space in which the imaging device is present or a flying object that flies in the real space.

13. The information processing device according to claim 1 , wherein the imaging device is configured to be attached onto a part of a body of a person who is different from a user of the display terminal.

14. The information processing device according to claim 13 ,
wherein the imaging device includes a plurality of cameras, and
wherein the captured image includes a plurality of captured images obtained from the plurality of cameras.

15. The information processing device according to claim 14 , wherein the imaging device comprises a wearable camera configured to communicate with the display terminal via the network server in real time.

16. The information processing device according to claim 15 ,
wherein the plurality of captured images is a plurality of circumferential captured images each of which represents a circumference of the wearable camera, and
wherein the circuitry is further configured to select, based on the view information, at least one of the plurality of circumferential captured images as the partial image without performing collation of feature points between the plurality of the captured images.

17. The information processing device according to claim 1 , wherein the circuitry is further configured to control the display terminal to change, in the correction on the captured image, the partial image to gradually follow a rotation movement of the imaging device.

18. The information processing device according to claim 1 , wherein the circuitry is further configured to:
determine whether an imaging direction of the imaging device is changed while a position of the imaging device in a real space is not substantially changed; and
perform the correction on perform the correction on the captured image so as to suppress the change in the captured image based on determination that the imaging direction of the imaging device is changed while the position of the imaging device in the real space is not substantially changed.

19. The information processing device according to claim 18 , wherein the circuitry is further configured to determine, based on a magnitude of a rotation angle in accordance with a change of the imaging direction, whether the imaging direction of the imaging device is changed while the position of the imaging device.

20. An information processing method comprising:
controlling a display terminal to:
acquire a captured image obtained by an imaging device via a network server;
display a partial image of the captured image in a display region of the display terminal, wherein
the partial image corresponds to a

Assignees

• Sony Corp

Inventors

• Rekimoto Junichi
• Kasahara Shunichi

Classifications

• H04N7/185 Primary

  from a mobile camera, e.g. for remote control · CPC title

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/14

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title