Screen interaction using EOG coordinates

Patent metadata

Field	Value
Publication number	US-12405662-B2
Application number	US-202218290673-A
Country	US
Kind code	B2
Filing date	Jul 21, 2022
Priority date	Jul 21, 2021
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

A method comprising acquiring a set of voltage signals from a set of electrodes arranged in proximity to the ears of a user, based on the set of voltage signals, determining an EOG gaze vector in ego-centric coordinates, determining a head pose of the user in display coordinates, using a sensor device worn by the user, combining the EOG gaze vector and head pose to obtain a gaze vector in display coordinates, and determining a gaze point by calculating an intersection of the gaze vector and an imaging surface having a known position in display coordinates.

First claim

The invention claimed is:

1. A method comprising:
acquiring a set of voltage signals from a set of electrodes arranged in proximity to ears of a user;
based on the set of voltage signals, determining an electro-oculography (EOG) gaze vector in ego-centric coordinates;
determining a head pose of the user in display coordinates, using a sensor device worn by the user;
combining the EOG gaze vector and the head pose to obtain a gaze vector in display coordinates; and
determining a gaze point by calculating an intersection of the gaze vector and an imaging surface having a known position in display coordinates.

2. The method according to claim 1 , further comprising:
calibrating the sensor device to obtain a position of the sensor device in display coordinates.

3. The method according to claim 2 , wherein the calibrating includes:
displaying a graphical element on the imaging surface; and
receiving user input confirming the user is looking at a location on the imaging surface corresponding to the graphical element.

4. The method according to claim 2 , wherein the user-worn sensor device is synchronized with a second sensor device fixedly arranged relative to at least a portion of a display system including the imaging surface.

5. The method according to claim 2 , wherein the calibrating includes:
determining a distance between the user and the imaging surface, optionally using one or several sensors in the display system.

6. The method according claim 1 , further comprising: performing an on-line statistical calibration including statistical analysis of the gaze point over time and knowledge of expected user viewing patterns.

7. The method of claim 1 , further comprising:
receiving image data for display on the imaging surface;
determining a gaze depth based at least in part on depth information associated with the gaze point;
calculating a relative depth associated with a first portion of the image data as the difference between depth information associated with the first portion of the image data and the gaze depth; and
generating modified image data by modifying pixel data associated with the first portion of the image data in accordance with a function of the relative depth associated with the first portion of the image data.

8. The method according to claim 7 , wherein modifying pixel data involves altering one or more of: hue, brightness, gamma, contrast, sharpness, blur, and spatial filtering of the pixel data.

9. The method of claim 1 , further comprising:
receiving image data for display on the imaging surface;
determining a gaze depth based at least in part on depth information associated with the gaze point;
based at least in part on the gaze point and the gaze depth, identifying at least one audio object associated with a current point of interest; and
modifying the audio objects such that the identified audio objects are modified differently than other audio objects.

10. The method according to claim 9 , wherein modifying the identified audio objects involves altering one of: volume, loudness, and frequency distribution of the identified audio objects.

11. The method of claim 1 , further comprising:
receiving image data for display on the imaging surface;
monitoring over time the gaze point on the imaging surface;
determining an average gaze position and a gaze radius;
comparing the gaze radius with a radius threshold; and
in accordance with a determination that the gaze radius is smaller than the radius threshold, applying a zoom of the image data.

12. The method according to claim 11 , further comprising:
determining a smallest distance between the average gaze position and one or more edges of the imaging surface;
comparing the smallest distance with a distance threshold; and
in accordance with a determination that the smallest distance is less than the distance threshold, applying an offset off the image data to increase the smallest distance.

13. The method according to claim 11 , further comprising: low-pass filtering the offset or the zoom before application to the image data.

14. A system comprising:
a set of electrodes arranged in proximity to ears of a user and configured to acquire a set of voltage signals;
an electro-oculography (EOG) processing unit for determining an EOG gaze vector in ego-centric coordinates based on the set of voltage signals;
a user-worn sensor device for determining a head pose of the user in display coordinates;
a processing unit configured to:
combine the EOG gaze vector and the head pose to obtain a gaze vector in display coordinates; and
determine a gaze point by calculating an intersection of the gaze vector and an imaging surface having a known position in display coordinates.

15. A non-transitory computer readable medium storing computer program code configured to perform, when executed on a computer processor, the steps:
acquiring a set of voltage signals from a set of electrodes arranged in proximity to ears of a user;
based on the set of voltage signals, determining an electro-oculography (EOG) gaze vector in ego-centric coordinates;
determining a head pose of the user in display coordinates, using a sensor device worn by the user;
combining the EOG gaze vector and the head pose to obtain a gaze vector in display coordinates; and
determining a gaze point by calculating an intersection of the gaze vector and an imaging surface having a known position in display coordinates.

Assignees

• Dolby Laboratories Licensing Corp

Inventors

• Fanelli Andrea
• Gitterman Evan David
• Swedlow Nathan Carl
• Brandmeyer Alex
• Joyner Mcgregor Steele
• Daly Scott
• Crum Poppy Anne Carrie

Classifications

• H04S7/303

  Tracking of listener position or orientation · CPC title

• G06F2203/04806

  Zoom, i.e. interaction techniques or interactors for controlling the zooming operation · CPC title

• G06F3/0487

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title