Virtual object repositioning versus motion of user and perceived or expected delay

Patent metadata

Field	Value
Publication number	US-11263456-B2
Application number	US-201816757974-A
Country	US
Kind code	B2
Filing date	Aug 9, 2018
Priority date	Oct 31, 2017
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

Provided is an information processing device that includes a position acquisition section that acquires a position of a virtual object in a real space, having been determined on the basis of a recognition result of a real space corresponding to an image captured by an imaging device, and a display control section that controls display of at least a boundary of the virtual object, on the basis of the position of the virtual object, and a feature of a real object that resides in the real space and is away from the position of the virtual object.

First claim

The invention claimed is:

1. An information processing device, comprising:
a central processing unit (CPU) configured to:
determine a virtual object in a real space based on a recognition result of the real space corresponding to an image captured by an imaging device;
acquire a first position of the virtual object in the real space based on the determination of the virtual object;
calculate a first degree of delay in display of the virtual object based on a feature of a real object in the real space; and
control display of at least a boundary of the virtual object at a second position away from the first position with respect to the real object, wherein
the display of at least the boundary of the virtual object is controlled based on
the acquired first position of the virtual object, and
the calculated first degree of delay in the display of the virtual object is greater than a first value, and

a distance between the acquired first position and the real object is smaller than a second value.

2. The information processing device of claim 1 , wherein
in a case where the virtual object and the real object are in a first positional relation, the CPU is further configured to control display of at least the boundary of the virtual object in a manner different from a case where the virtual object and the real object are in a second positional relation which is different from the first positional relation.

3. The information processing device of claim 2 , wherein
in a case where positional relation between the virtual object and the real object is changed from the first positional relation to the second positional relation, in response to a motion of a field of view of a user, the CPU is further configured to modify the display of at least the boundary of the virtual object, and
the user is associated with the information processing device.

4. The information processing device of claim 1 , wherein
the CPU is further configured to modify a shape of the virtual object based on the first degree of delay in the display of the virtual object is greater than the first value.

5. The information processing device of claim 1 , wherein
the CPU is further configured to one of give a motion to the virtual object or increase the motion of the virtual object based on the first degree of delay in the display of the virtual object is greater than the first value.

6. The information processing device of claim 1 , wherein
the CPU is further configured to modify at least one of color or luminance of the virtual object based on the first degree of delay in the display of the virtual object, and
the at least one of the color or the luminance of the virtual object is modified to be similar to at least one of color or luminance of the real object.

7. The information processing device of claim 1 , wherein
the CPU is further configured to reduce recognizability of the boundary of the virtual object based on the first degree of delay in the display of the virtual object.

8. The information processing device of claim 1 , wherein
the feature of the real object includes at least one of shape of the real object or motion of the real object.

9. The information processing device of claim 1 , wherein
the CPU is further configured to calculate the first degree of delay in the display of the virtual object based on a feature of the virtual object.

10. The information processing device of claim 9 , wherein
the feature of the real object includes a position of a straight line that composes the real object,
the feature of the virtual object includes the first position of the virtual object,
the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and
the position of the virtual object and the position of the straight line that composes the real object is closer corresponding to the first degree of delay than the second degree of delay.

11. The information processing device of claim 9 , wherein
the feature of the real object includes a direction of a straight line that composes the real object,
the feature of the virtual object includes a direction of a straight line that composes the virtual object,
the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and
an angle between the direction of the straight line that composes the virtual object and the direction of the straight line that composes the real object is smaller corresponding to the first degree of delay than the second degree of delay.

12. The information processing device of claim 9 ,
wherein the feature of the virtual object includes at least one of a shape of the virtual object or motion of the virtual object.

13. The information processing device of claim 9 , wherein
the feature of the real object includes a color of the real object,
the feature of the virtual object includes a color of the virtual object,
the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and
the color of the real object and the color of the virtual object is less similar corresponding to the first degree of delay than the second degree of delay.

14. The information processing device of claim 9 , wherein
the feature of the real object includes luminance of the real object,
the feature of the virtual object includes luminance of the virtual object,
the CPU is further configured to calculate the first degree of delay in the display of the virtual object to be greater than a second degree of delay in the display of the virtual object, and
a difference between the luminance of the real object and the luminance of the virtual object is larger corresponding to the first degree of delay than the second degree of delay.

15. The information processing device of claim 1 , wherein
the CPU is further configured to:
calculate motion of a virtual camera from the recognition result of the real space; and
control display of at least the boundary of the virtual object based on the calculated motion of the virtual camera.

16. The information processing device of claim 15 , wherein
the CPU is further configured to modify the display of at least the boundary of the virtual object based on the motion of the virtual camera exceeds a threshold motion value.

17. An information processing method, comprising:
determining a position of a virtual camera based on a recognition result of a real space corresponding to an image captured by an imaging device;
determining a virtual object in the real space based on the determined position of the virtual camera;
acquiring a first position of the virtual object in the real space based on the determination of the virtual object;
calculating a degree of delay in display of the virtual object based on a feature of a real object in the real space; and
controlling display of at least a boundary of the virtual object at a second position away from the first position with respect to the real object, wherein
the display of at least the boundary of the virtual object is controlled based on
the acquired first position of the virtual object, and
the calculated degree of delay in the display of the virtual object is greater than a first value, and

a distance between the acquired first position and the real obj

Assignees

• Sony Corp

Inventors

• Ishihara Atsushi

Classifications

• G06V40/103 Primary

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title