Information processing apparatus and information processing method

Patent metadata

Field	Value
Publication number	US-10452128-B2
Application number	US-201615560111-A
Country	US
Kind code	B2
Filing date	Feb 18, 2016
Priority date	Mar 31, 2015
Publication date	Oct 22, 2019
Grant date	Oct 22, 2019

Abstract

An information processing apparatus including circuitry configured to acquire information indicating a spatial relationship between a real object and a virtual object, and initiate generation of a user feedback based on the acquired information, the user feedback being displayed to be augmented to a generated image obtained based on capturing by an imaging device, or augmented to a perceived view of the real world, and wherein a characteristic of the user feedback is changed when the spatial relationship between the real object and the virtual object changes.

First claim

The invention claimed is:

1. An information processing apparatus, comprising:
a sensor configured to acquire positional information of a real object in front of a user of a head-mounted display, the real object being operated by the user in a real space; and
a control unit configured to:
access a program that is recorded in a memory for the head-mounted display;
control, based on the accessed program, the head-mounted display to display a first virtual object over a view of a real world;
acquire the positional information from the sensor;
determine, based on the positional information, a spatial relationship between the real object and the first virtual object;
control the head-mounted display to display a second virtual object over the first virtual object based on the spatial relationship, wherein a position of the second virtual object corresponds to a position of the real object; and
control the head-mounted display to change display of the second virtual object when the spatial relationship between the real object and the first virtual object changes.

2. The information processing apparatus according to claim 1 ,
wherein the spatial relationship represents a distance between a hand of the user and the first virtual object, and
wherein the control unit is further configured to control the head-mounted display to change a size of the display of the second virtual object as the distance between the real object and the first virtual object is changed.

3. The information processing apparatus according to claim 2 , wherein the control unit is further configured to control the head-mounted display to decrease a size of the second virtual object as the distance between the real object and the first virtual object decreases.

4. The information processing apparatus according to claim 2 , wherein the control unit is further configured to control the head-mounted display to change a shape of the second virtual object based on the distance between the real object and the first virtual object and a 3 D shape of the first virtual object.

5. The information processing apparatus according to claim 1 , wherein the control unit is further configured to control the head-mounted display to change a visual quality of the second virtual object as the spatial relationship between the real object and the first virtual object.

6. The information processing apparatus according to claim 5 , wherein the control unit is further configured to control the head-mounted display to change at least one of a color, a brightness, or transmittance of display of the second virtual object as the spatial relationship between the real object and the first virtual object is changed.

7. The information processing apparatus according to claim 6 , wherein the control unit is further configured to control the head-mounted display to set the color of the second virtual object lighter as the real object becomes closer to the first virtual object.

8. The information processing apparatus according to claim 1 , wherein the control unit is further configured to control the head-mounted display to move a position of the second virtual object as the spatial relationship between the real object and the first virtual object is changed.

9. The information processing apparatus according to claim 1 , wherein at least a portion of the first virtual object is not displayed as the real object approaches the first virtual object.

10. The information processing apparatus according to claim 1 , wherein at least a portion of the first virtual object is lighted up as the real object approaches the first virtual object.

11. The information processing apparatus according to claim 1 , wherein the spatial relationship includes a distance between the real object and the first virtual object.

12. The information processing apparatus according to claim 11 , wherein a characteristic of the second virtual object is changed based on whether the real object is within a predetermined distance from the first virtual object.

13. The information processing apparatus according to claim 12 , wherein the characteristic of the second virtual object which is changed in a case that the real object is within the predetermined distance from the first virtual object is different from a characteristic of the second virtual object which is changed in a case that the real object is not within the predetermined distance from the first virtual object.

14. The information processing apparatus according to claim 12 , wherein the control unit is further configured to set the predetermined distance in front of and behind the first virtual object as a range of operation of the first virtual object.

15. The information processing apparatus according to claim 1 , wherein the first virtual object overlaps the real object when the real object is pushed through a location corresponding to a display position of the first virtual object.

16. The information processing apparatus according to claim 1 , wherein a warning feedback is given when the real object is pushed through a location corresponding to a display position of the first virtual object.

17. The information processing apparatus according to claim 1 , wherein display of at least a portion of the first virtual object is terminated when the real object is pushed through a location corresponding to a display position of the first virtual object.

18. The information processing apparatus according to claim 1 , wherein the second virtual object includes a display portion for an input operation performed by the user.

19. The information processing apparatus according to claim 1 , wherein the spatial relationship includes at least one of a distance, a direction, or posture information between the real object and the first virtual object.

20. The information processing apparatus according to claim 1 , wherein a characteristic of the second virtual object is changed stepwise or in relation to a changing of the spatial relationship between the real object and the first virtual object.

21. The information processing apparatus according to claim 1 , wherein a magnitude of a characteristic of the second virtual object is changed in relationship to a changing of the spatial relationship between the real object and the first virtual object.

22. The information processing apparatus according to claim 1 ,
wherein the head-mounted display is an optical see-through head mounted display, and
wherein the control unit is further configured to control the optical see-through head mounted display to display the first virtual object and the second virtual object over the view of the real world.

23. The information processing apparatus according to claim 22 , wherein the control unit is further configured to control the optical see-through head mounted display to display the first virtual object and the second virtual object stereoscopically.

24. The information processing apparatus according to claim 1 ,
wherein the head-mounted display is a smartphone,
wherein the sensor is an imaging device that is installed in the smartphone and configured to capture the real world, and
wherein the control unit is further configured to control the smartphone to display the view of the real world, the first virtual object, and the second virtual object simultaneously.

25. An information processing method, comprising:
controlling a mobile display terminal to display a first virtual object

Assignees

• Sony Corp

Inventors

• Ishihara Atsushi
• Ishikawa Tsuyoshi
• Aga Hiroyuki
• Kawasaki Koichi
• Nishibe Mitsuru
• Kusano Yuji

Classifications

• G09G2340/14

  Solving problems related to the presentation of information to be displayed · CPC title

• G09G3/003

  to produce spatial visual effects · CPC title

• G06F3/147 Primary

  using display panels · CPC title

• G09G2354/00

  Aspects of interface with display user · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title