Methods and systems for selection of objects

Patent metadata

Field	Value
Publication number	US-11475642-B2
Application number	US-202017127022-A
Country	US
Kind code	B2
Filing date	Dec 18, 2020
Priority date	Dec 18, 2020
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

Methods and systems for selecting an object or location in an extended reality (XR) environment or physical environment are described. A first origin, including a first position and a first direction, and a second origin, including a second position and a second direction, are obtained by at least one sensor. An intersection of a first ray, casted from the first origin, and a second ray, casted from the second origin, is determined. A selected object or selected location is identified, based on the determined intersection. An identification of the selected object or the selected location is outputted.

First claim

The invention claimed is:

1. A method for selecting an object or a location, the method comprising:
obtaining, by at least one sensor, a first origin, including a first position and a first direction, and a second origin, including a second position and a second direction;
obtaining an intersection between a first ray casted from the first position and from the first direction of the first origin and a second ray casted from the second position and from the second direction of the second origin, wherein obtaining the intersection between the first ray and the second ray comprises computing a projected intersection point between the first ray and the second ray, the first and second rays being skew, wherein computing the projected intersection point comprises:
computing a shortest vector between the first ray and the second ray; and
computing a point along the shortest vector as the projected intersection point;

identifying a selected object or selected location, based on the obtained intersection between the first ray and the second ray; and
outputting an identification of the selected object or the selected location.

2. The method of claim 1 , wherein the selected object or the selected location is in an extended reality (XR) environment, the method further comprising:
identifying one or more intersected virtual objects in the XR environment that are intersected by at least one of the first ray or the second ray; and
adjusting opacity of the one or more intersected virtual objects, to render the one or more intersected virtual objects translucent.

3. The method of claim 1 , wherein obtaining the intersection between the first ray and the second ray comprises computing an actual intersection point between the first ray and the second ray.

4. The method of claim 1 , further comprising:
computing the point along the shortest vector after determining that a magnitude of the shortest vector is within a predefined threshold.

5. The method of claim 4 , wherein the predefined threshold is relative to a distance between at least one of the first origin or the second origin to the projected intersection point.

6. The method of claim 1 , wherein identifying the selected object or the selected location comprises:
identifying one or more provisionally selected objects or provisionally selected locations, based on the obtained intersection between the first ray and the second ray; and
receiving confirmation of one selected object or one selected location from among the one or more provisionally selected objects or provisionally selected locations.

7. The method of claim 6 , wherein identifying one or more provisionally selected objects or provisionally selected locations comprises:
projecting a sphere with an adjustable predetermined radius at the obtained intersection between the first ray and the second ray; and
identifying the one or more provisionally selected objects or provisionally selected locations coinciding with the projected sphere.

8. The method of claim 1 , wherein the first origin is obtained by the at least one sensor based on a tracked first position, orientation or pose of a first body part of a user, and wherein the second origin is obtained by at least one same or different sensor based on a tracked second position, orientation or pose of a second body part of the user.

9. The method of claim 1 , wherein the selected object is a virtual object rendered in an extended reality (XR) environment, is a real-world object in the XR environment; or is a real-world object in a physical environment.

10. The method of claim 1 , wherein the at least one sensor includes at least one of:
a sensor of a handheld device;
a sensor of a wearable device;
a sensor of a smart device; or
a sensor of a tracking system.

11. A computing system comprising:
a processing device coupled to a memory storing machine-executable instructions thereon, wherein the instructions, when executed by the processing device, cause the computing system to:
obtain, by at least one sensor, a first origin, including a first position and a first direction, and a second origin, including a second position and a second direction;
obtain an intersection between a first ray casted from the first position and from the first direction of the first origin and a second ray casted from the first position and from the second direction of the second origin, wherein obtaining the intersection between the first ray and the second ray comprises computing a projected intersection point between the first ray and the second ray, the first and second rays being skew, wherein computing the projected intersection point comprises:
computing a shortest vector between the first ray and the second ray; and
computing a point along the shortest vector as the projected intersection point;

identify a selected object or selected location, based on the obtained intersection between the first ray and the second ray; and
output an identification of the selected object or the selected location.

12. The computing system of claim 11 , wherein the selected object or the selected location is in an extended reality (XR) environment, and wherein the instructions further cause the computing system to:
identify one or more intersected virtual objects in the XR environment that are intersected by at least one of the first ray or the second ray; and
adjust opacity of the one or more intersected virtual objects, to render the one or more intersected virtual objects translucent.

13. The computing system of claim 11 , wherein the instructions cause the computing system to obtain the intersection between the first ray and the second ray by computing an actual intersection point of the first ray and the second ray.

14. The computing system of claim 11 , wherein the instructions cause the computing system to obtain the intersection between the first ray and the second ray by computing a projected intersection point of the first ray and the second ray, the first and second rays being skew.

15. The computing system of claim 11 , wherein the first origin is obtained by the at least one sensor based on a tracked first position, orientation or pose of a first body part of a user, and wherein the second origin is obtained by at least one same or different sensor based on a tracked second position, orientation or pose of a second body part of the user.

16. The computing system of claim 11 , wherein the selected object is a virtual object rendered in an extended reality (XR) environment, is a real-world object in the XR environment, or is a real-world object in a physical environment.

17. The computing system of claim 11 , wherein the computing system is one of:
a head-mounted display (HMD) device;
a wearable device;
a handheld device;
a smart device;
a tracking system; or
a desktop device.

18. A non-transitory computer-readable medium having machine-executable instructions stored thereon, the instructions, when executed by a processing device of a computing system, cause the computing system to:
obtain, by at least one sensor, a first origin, including a first position and a first direction, and a second origin, including a second position and a second direction;
obtain an intersection between a first ray casted from the first position and from the first direction of the first origin and a second ray casted from the second position and from the second direction of the second origin, wherein obtaining the intersection between the first ray and the second ray comprises computing a projected intersection poi

Assignees

• Fan Szu Wen
• Khan Taslim Arefin
• Li Wei
• Huawei Tech Co Ltd

Inventors

• Fan Szu Wen
• Khan Taslim Arefin
• Li Wei

Classifications

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06T15/06

  Ray-tracing · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title