Systems and methods for responsive user interface based on gaze depth

Patent metadata

Field	Value
Publication number	US-2026093323-A1
Application number	US-202519413167-A
Country	US
Kind code	A1
Filing date	Dec 9, 2025
Priority date	Dec 9, 2022
Publication date	Apr 2, 2026
Grant date	—

Abstract

In virtual reality (VR) and augmented reality (AR), eye tracking may be performed to determine the user's gaze direction. The gaze direction may be used to enhance user interaction. However, when a user gazes in a particular direction, it could sometimes be the case that there are multiple items located in that gaze direction, each at a different depth. The gaze of direction alone might not be indicative of the item at which the user is looking. Therefore, in some embodiments, to try to further enhance user interaction, a gaze depth of the gaze may be determined. Some embodiments are directed to performing eye tracking to detect a gaze depth of a human's gaze and modifying a user interface (UI) responsive to a change in the gaze depth.

First claim

1 . A computer-implemented method comprising:
generating a user interface for overlay onto a view of a real-world or virtual space; performing first eye tracking to detect that a gaze of a human is focused on a first object in the view; subsequently performing second eye tracking to detect that the gaze of the human is instead now focused on a second object in the view, wherein the second object is different from the first object; responsive to detecting that the gaze of the human is instead now focused on the second object in the view, modifying the user interface to include content that is dependent upon the gaze of the human previously being focused on the first object in the view.

2 . The computer-implemented method of claim 1 , wherein the content references the first object.

3 . The computer-implemented method of claim 1 , wherein the content is dependent upon both the first object and the second object.

4 . The computer-implemented method of claim 3 , wherein the content draws a comparison between the first object and the second object.

5 . The computer-implemented method of claim 1 , wherein modifying the user interface includes modifying a size of the user interface.

6 . The computer-implemented method of claim 1 , wherein the first object and the second object are in different gaze directions, wherein performing the first eye tracking includes detecting a first gaze direction corresponding to the first object, and wherein performing the second eye tracking includes detecting a second gaze direction corresponding to the second object.

7 . The computer-implemented method of claim 1 , wherein the first object and the second object are at different gaze depths, wherein performing the first eye tracking includes detecting a first gaze depth corresponding to the first object, and wherein performing the second eye tracking includes detecting a second gaze depth corresponding to the second object.

8 . The computer-implemented method of claim 7 , wherein a gaze depth is detected by:
determining a first vector representing a gaze direction of a left eye; determining a second vector representing a gaze direction of a right eye; and determining the gaze depth based on convergence of the first vector and the second vector.

9 . The computer-implemented method of claim 1 , wherein the view is rendered on a display.

10 . The computer-implemented method of claim 1 , wherein the view is of the real-world space seen through a display that is at least partially transparent.

11 . A system comprising:
at least one processor; and a memory storing processor-executable instructions that, when executed by the at least one processor, cause the system to:
generate a user interface for overlay onto a view of a real-world or virtual space;
perform first eye tracking to detect that a gaze of a human is focused on a first object in the view;
subsequently perform second eye tracking to detect that the gaze of the human is instead now focused on a second object in the view, wherein the second object is different from the first object;
responsive to detecting that the gaze of the human is instead now focused on the second object in the view, modify the user interface to include content that is dependent upon the gaze of the human previously being focused on the first object in the view.

12 . The system of claim 11 , wherein the content references the first object.

13 . The system of claim 11 , wherein the content is dependent upon both the first object and the second object.

14 . The system of claim 13 , wherein the content draws a comparison between the first object and the second object.

15 . The system of claim 11 , wherein modifying the user interface includes modifying a size of the user interface.

16 . The system of claim 11 , wherein the first object and the second object are in different gaze directions, wherein performing the first eye tracking includes detecting a first gaze direction corresponding to the first object, and wherein performing the second eye tracking includes detecting a second gaze direction corresponding to the second object.

17 . The system of claim 11 , wherein the first object and the second object are at different gaze depths, wherein performing the first eye tracking includes detecting a first gaze depth corresponding to the first object, and wherein performing the second eye tracking includes detecting a second gaze depth corresponding to the second object.

18 . The system of claim 17 , wherein a gaze depth is detected by:
determining a first vector representing a gaze direction of a left eye; determining a second vector representing a gaze direction of a right eye; and determining the gaze depth based on convergence of the first vector and the second vector.

19 . The system of claim 11 , wherein the view is rendered on a display.

20 . A non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
generating a user interface for overlay onto a view of a real-world or virtual space; performing first eye tracking to detect that a gaze of a human is focused on a first object in the view; subsequently performing second eye tracking to detect that the gaze of the human is instead now focused on a second object in the view, wherein the second object is different from the first object; responsive to detecting that the gaze of the human is instead now focused on the second object in the view, modifying the user interface to include content that is dependent upon the gaze of the human previously being focused on the first object in the view.

Assignees

• Shopify Inc

Inventors

• Maschmeyer Russ
• Florenzano Eric Andrew
• Letkeman Brennan
• Bello Diego Macario
• Beauchamp Daniel

Classifications

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F2203/04804

  Transparency, e.g. transparent or translucent windows · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title