Rerendering a position of a hand to decrease a size of a hand to create a realistic virtual/augmented reality environment

Patent metadata

Field	Value
Publication number	US-11954808-B2
Application number	US-202217666533-A
Country	US
Kind code	B2
Filing date	Feb 7, 2022
Priority date	Sep 8, 2015
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

The technology disclosed relates to a method of realistic rendering of a real object as a virtual object in a virtual space using an offset in the position of the hand in a three-dimensional (3D) sensory space. An offset between expected positions of the eye(s) of a wearer of a head mounted device and a sensor attached to the head mounted device for sensing a position of at least one hand in a three-dimensional (3D) sensory space is determined. A position of the hand in the three-dimensional (3D) sensory space can be sensed using a sensor. The sensed position of the hand can be transformed by the offset into a re-rendered position of the hand as would appear to the wearer of the head mounted device if the wearer were looking at the actual hand. The re-rendered hand can be depicted to the wearer of the head mounted device.

First claim

What is claimed is:

1. A system including:
a processor; and
a memory storing computer instructions that, when executed by the processor, perform operations comprising:
applying an obtained transformation matrix to a sensed positon of a control object,
wherein the obtained transformation matrix is calculated based on an actual position of a sensor that is head-mounted and for decreasing a rendered size of the sensed control object (i) in accordance with an identified offset between an eye of a user and the actual position of the sensor (ii) and with respect to the sensed control object being rendered without consideration of the offset, and
wherein the applied transformation matrix transforms the sensed position of the control object into a re-rendered position of the control object, the re-rendered position of the control object placing the control object such that the user perceives the re-rendered position of the control object; and

providing for display, to the user, a re-rendered control object image using the re-rendered position of the control object.

2. The system of claim 1 , further configured to perform:
sensing additional positions of the control object and determining a motion or gesture from the sensed additional positions; and
applying the offset to the additional positions of the control object to transform the sensed additional positions to re-rendered positions of the control object as seen by the user, thereby providing the re-rendered control object image having the motion or gesture of the control object.

3. The system of claim 2 , further configured to perform:
sensing an indication of quality of tracking for the sensor, the indication of quality of tracking indicating whether the sensor is tracking a position or motion of the control object; and
adding to the re-rendered control object image a visual indication corresponding to the indication of quality of tracking, thereby providing the user with a visual cue of how well the sensor is tracking the control object.

4. The system of claim 3 , wherein the indication of quality of tracking includes an indication that the sensor is able to recognize the control object in captured images.

5. The system of claim 4 , wherein the visual indication corresponding to the indication of quality of tracking includes a colored outline circumscribed about the re-rendered control object image when the sensor is able to recognize the control object in captured images.

6. The system of claim 4 , wherein the visual indication corresponding to the indication of quality of tracking includes a colored outline circumscribed about the re-rendered control object image when the sensor is unable to recognize the control object in captured images.

7. A method comprising:
applying an obtained transformation matrix to a sensed position of a control object,
wherein the obtained transformation matrix is calculated based on an actual position of a sensor that is head-mounted and for decreasing a rendered size of the sensed control object (i) in accordance with an identified offset between an eye of a user and the actual position of the sensor (ii) and with respect to the sensed control object being rendered without consideration of the offset, and
wherein the applied transformation matrix transforms the sensed position of the control object into a re-rendered position of the control object, the re-rendered position of the control object placing the control object such that the user perceives the re-rendered position of the control object; and

providing for display, to the user, a re-rendered control object image using the re-rendered position of the control object.

8. The method of claim 7 , further including:
sensing additional positions of the control object and determining a motion or gesture from the sensed additional positions; and
applying the offset to the additional positions of the control object to transform the sensed additional positions to re-rendered positions of the control object as seen by the user, thereby providing the re-rendered control object image having the motion or gesture of the control object.

9. The method of claim 8 , further including:
sensing an indication of quality of tracking for the sensor, the indication of quality of tracking indicating whether the sensor is tracking a position or motion of the control object; and
adding to the re-rendered control object image a visual indication corresponding to the indication of quality of tracking, thereby providing the user with a visual cue of how well the sensor is tracking the control object.

10. The method of claim 9 , wherein the indication of quality of tracking includes an indication that the sensor is able to recognize the control object in captured images.

11. The method of claim 10 , wherein the visual indication corresponding to the indication of quality of tracking includes a colored outline circumscribed about the re-rendered control object image when the sensor is able to recognize the control object in captured images.

12. The method of claim 10 , wherein the visual indication corresponding to the indication of quality of tracking includes a colored outline circumscribed about the re-rendered control object image when the sensor is unable to recognize the control object in captured images.

13. A non-transitory computer-readable recording medium having instructions recorded thereon, the instructions, when executed by a processor, perform a method comprising:
applying an obtained transformation matrix to a sensed position of a control object,
wherein the obtained transformation matrix is calculated based on an actual position of a sensor that is head-mounted and for decreasing a rendered size of the sensed control object (i) in accordance with an identified offset between an eye of a user and the actual position of the sensor (ii) and with respect to the sensed control object being rendered without consideration of the offset, and
wherein the applied transformation matrix transforms the sensed position of the control object into a re-rendered position of the control object, the re-rendered position of the control object placing the control object such that the user perceives the re-rendered position of the control object; and

providing for display, to the user, a re-rendered control object image using the re-rendered position of the control object.

14. The non-transitory computer-readable recording medium of claim 13 , wherein the method further comprises:
sensing additional positions of the control object and determining a motion or gesture from the sensed additional positions; and
applying the offset to the additional positions of the control object to transform the sensed additional positions to re-rendered positions of the control object as seen by the user, thereby providing the re-rendered control object image having the motion or gesture of the control object.

15. The non-transitory computer-readable recording medium of claim 14 , wherein the method further comprises:
sensing an indication of quality of tracking for the sensor, the indication of quality of tracking indicating whether the sensor is tracking a position or motion of the control object; and
adding to the re-rendered control object image a visual indication corresponding to the indication of quality of tracking, thereby providing the user with a visual cue of how well the sensor is tracking the control object.

16. The non-transitory computer-readable recording medium of claim 15 , wherein the indication of quality of tracking includes an indication that the sensor is abl

Assignees

• Ultrahaptics Ip Two Ltd

Inventors

• Marcolina Alex
• Holz David

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title