Adding, placing, and grouping widgets in extended reality (xr) applications

Patent metadata

Field	Value
Publication number	US-2025004606-A1
Application number	US-202418752157-A
Country	US
Kind code	A1
Filing date	Jun 24, 2024
Priority date	Jun 27, 2023
Publication date	Jan 2, 2025
Grant date	—

Abstract

A method includes rendering a semi-transparent or opaque board on a display of an XR headset, such that the board does not overlap with at least one real-world object. The method also includes receiving a first user input to open a widget library. The method also includes rendering a grid structure on the board after receiving the first user input. The method also includes receiving a second user input to select a widget from the widget library. The method also includes receiving a third user input to move the selected widget to the grid structure on the board. The method also includes placing the selected widget at a position in the grid structure on the board. The method also includes stopping rendering the grid structure, while displaying the selected widget at the position on the board, with an orientation determined based at least partly on a user's head position.

First claim

What is claimed is:

1 . A method comprising:
rendering, by a processor, a semi-transparent or opaque board on a display of an extended reality (XR) headset in communication with the processor, such that the board does not overlap with at least one real-world object recognized by the processor; receiving, by the processor, a first user input to open a widget library; rendering, by the processor, a grid structure on the board after receiving the first user input; receiving, by the processor, a second user input to select a widget from the widget library; receiving, by the processor, a third user input to move the selected widget to the grid structure on the board; placing, by the processor, the selected widget at a position in the grid structure on the board; and stopping, by the processor, rendering the grid structure, while displaying the selected widget at the position on the board, with an orientation determined based at least partly on a user's head position.

2 . The method of claim 1 , wherein the grid structure includes a dot grid structure or a line grid structure.

3 . The method of claim 1 , wherein the second user input includes a pinch gesture, and the third user input includes a drag gesture while maintaining the pinch gesture.

4 . The method of claim 1 , wherein placing the selected widget at the position comprises moving one or more pre-existing widgets within the grid structure.

5 . The method of claim 1 , wherein placing the selected widget at the position comprises:
highlighting one or more candidate grid positions for the selected widget; and visually indicating one or more locations occupied by one or more virtual or real-world objects recognized by the processor as obstacles.

6 . The method of claim 1 , wherein:
the second user input and the third user input comprise images of the user's eyes captured using an eye tracking camera coupled to the XR headset; and the method further comprises:
detecting, based on the images from the eye tracking camera, that the user's eyes are focused on the selected widget.

7 . The method of claim 1 , wherein:
the second user input comprises images of the user's eyes captured using an eye tracking camera coupled to the XR headset; the third user input comprises a voice input from the user; and the method further comprises:
providing the voice input to a natural language processing module trained to identify keywords in the voice input including instructions to move the selected widget to the grid structure on the board.

8 . The method of claim 7 , wherein the keywords include a real-world object and a desired position for the widget relative to the real-world object.

9 . An electronic device comprising:
at least one processing device configured to:
render a semi-transparent or opaque board on a display of an extended reality (XR) headset in communication with the at least one processing device, such that the board does not overlap with at least one real-world object recognized by the at least one processing device;
receive a first user input to open a widget library;
render a grid structure on the board after receiving the first user input;
receive a second user input to select a widget from the widget library;
receive a third user input to move the selected widget to the grid structure on the board;
place the selected widget at a position in the grid structure on the board; and
stop rendering the grid structure while displaying the selected widget at the position on the board, with an orientation determined based at least partly on a user's head position.

10 . The electronic device of claim 9 , wherein the grid structure includes a dot grid structure or a line grid structure.

11 . The electronic device of claim 9 , wherein the second user input includes a pinch gesture, and the third user input includes a drag gesture while maintaining the pinch gesture.

12 . The electronic device of claim 9 , wherein to place the selected widget at the position, the at least one processing device is configured to move one or more pre-existing widgets within the grid structure.

13 . The electronic device of claim 9 , wherein to place the selected widget at the position, the at least one processing device is configured to:
highlight one or more candidate grid positions for the selected widget; and visually indicate one or more locations occupied by one or more virtual or real-world objects recognized by the at least one processing device as obstacles.

14 . The electronic device of claim 9 , wherein:
the second user input and the third user input comprise images of the user's eyes captured using an eye tracking camera coupled to the XR headset; and the at least one processing device is further configured to detect, based on the images from the eye tracking camera, that the user's eyes are focused on the selected widget.

15 . The electronic device of claim 9 , wherein:
the second user input comprises images of the user's eyes captured using an eye tracking camera coupled to the XR headset; the third user input comprises a voice input from the user; and the at least one processing device is further configured to provide the voice input to a natural language processing module trained to identify keywords in the voice input including instructions to move the selected widget to the grid structure on the board.

16 . The electronic device of claim 15 , wherein the keywords include a real-world object and a desired position for the widget relative to the real-world object.

17 . A non-transitory machine-readable medium containing instructions that when executed cause at least one processor of an electronic device to:
render a semi-transparent or opaque board on a display of an extended reality (XR) headset in communication with the at least one processor, such that the board does not overlap with at least one real-world object recognized by the at least one processor; receive a first user input to open a widget library; render a grid structure on the board after receiving the first user input; receive a second user input to select a widget from the widget library; receive a third user input to move the selected widget to the grid structure on the board; place the selected widget at a position in the grid structure on the board; and stop rendering the grid structure while displaying the selected widget at the position on the board, with an orientation determined based at least partly on a user's head position.

18 . The non-transitory machine-readable medium of claim 17 , wherein the grid structure includes a dot grid structure or a line grid structure.

19 . The non-transitory machine-readable medium of claim 17 , wherein the second user input includes a pinch gesture, and the third user input includes a drag gesture while maintaining the pinch gesture.

20 . The non-transitory machine-readable medium of claim 17 , wherein the instructions to place the selected widget at the position comprise instructions to move one or more pre-existing widgets within the grid structure.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Waghulde Ajinkya
• Evangelista Edgar Charles
• Horii Hiroshi
• Jiang Yujie
• Kim Sooryuh
• Cho Minkyung
• Shin Boosun
• Kim Sungman
• Lee Chaekyung

Classifications

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/0485

  Scrolling or panning · CPC title