Method, computer program and head mountable arrangement for assisting a subject to acquire spatial information about an environment

Patent metadata

Field	Value
Publication number	US-11586281-B2
Application number	US-202016795123-A
Country	US
Kind code	B2
Filing date	Feb 19, 2020
Priority date	Feb 19, 2019
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

A head mountable arrangement assists a subject to acquire spatial information about a surrounding environment by receiving gaze data from an eyetracker and spatial data from a distance sensor respectively. The gaze data describe an estimated point of regard of the subject, and the spatial data describe a distance between a reference point and an object in the surrounding environment. A feedback signal is provided, which indicates a distance from the subject to said object in the surrounding environment. The feedback signal is generated based on the estimated point of regard and the spatial data, and may reflect the distance to a surface element of an object that intersects a straight line between an eye-base line of the subject and the estimated point of regard, which surface element is located closer to the eye-base line than any other surface element of the objects in the surrounding environment along said straight line.

First claim

The invention claimed is:

1. A method performed in a processing circuitry for assisting a subject to acquire spatial information about a surrounding environment, comprising:
receiving gaze data describing an estimated point of regard of the subject;
receiving spatial data describing a distance between a reference point and an object in the surrounding environment;
generating a feedback signal for presentation to the subject, which feedback signal is generated based on the estimated point of regard and the spatial data, the feedback signal indicating a distance from the subject to said object in the surrounding environment; and
wherein the feedback signal is configured to cause at least one first sensory stimulus to be applied to the subject in a form of at least one of: an acoustic signal, a haptic signal and a visual signal,
wherein generating the feedback signal comprises presenting a visual signal providing an ocular cue to the subject confirming a position for the estimated point of regard at a particular object of said objects in the surrounding environment, the ocular cue being represented by at least one graphical element shown in a field of view for the subject at such a position that the at least one graphical element superimposes the estimated point of regard.

2. The method according to claim 1 , wherein the feedback signal indicates a movement of said object relative to the subject.

3. The method according to claim 1 , wherein an intensity of the at least one first sensory stimulus reflects a distance to a first surface element of an object intersecting a straight line between an eye-base line of the subject and the estimated point of regard, which first surface element is located closer to the eye-base line than any other surface element of the objects in the surrounding environment along said straight line.

4. The method according to claim 3 , wherein the intensity is relatively high for a comparatively short distance, and relatively low for a comparatively long distance.

5. The method according to claim 3 , wherein the reference point is located on the eye-base line.

6. The method according to claim 3 , wherein the reference point is located at a geometric center of a head mountable arrangement that incorporates an eyetracker configured to generate the gaze data, the geometric center representing an estimated point around which the head mountable arrangement is rotated when mounted on the subject.

7. The method according to claim 1 , wherein the spatial data describes color and light intensity information reflected from at least one object in the surrounding environment, and the method comprises:
obtaining at least one visual quality of at least one of said at least one object from the spatial data; and
generating the feedback signal such that it reflects said at least one visual quality.

8. The method according to claim 7 , wherein the at least one visual quality comprises at least one of a color, a surface texture and an object class describing said at least one object, and the feedback signal is configured to cause at least one second sensory stimulus to be applied to the subject, which at least one second sensory stimulus reflects at least one of the color, the surface texture and the object class and is different from the at least one first stimulus.

9. The method according to claim 1 , wherein generating the feedback signal comprises presenting an acoustic signal to the subject, which acoustic signal provides a sound cue confirming a position for the estimated point of regard at a particular object of said objects in the surrounding environment, the sound cue simulating an acoustic source located at the estimated point of regard via three-dimensional positional audio effects.

10. A processing circuitry configured to be included in a head mountable arrangement for assisting a subject to acquire spatial information about a surrounding environment, the processing circuitry comprising:
a first input interface configured to receive gaze data from an eyetracker, which gaze data describe an estimated point of regard of the subject;
a second input interface configured to receive spatial data describing a distance between a reference point and an object in the surrounding environment;
an output interface configured to provide a feedback signal indicating a distance from the subject to said object in the surrounding environment, the feedback signal being generated based on the estimated point of regard and the spatial data; and
wherein the output interface is configured to generate the feedback signal to cause at least one first sensory stimulus to be applied to the subject in a form of at least one of: an acoustic signal, a haptic signal and a visual signal,
wherein the output interface is configured to be connected to at least one display configured to present a visual signal to the subject, which visual signal provides an ocular cue confirming a position for the estimated point of regard at a particular object of said objects in the surrounding environment, the ocular cue being represented by at least one graphical element shown in a field of view for the subject at such a position that the at least one graphical element superimposes the estimated point of regard.

11. The processing circuitry according to claim 10 , wherein the feedback signal indicates a movement of said object relative to the subject.

12. The processing circuitry according to claim 11 , wherein the output interface is configured to be connected to at least one speaker configured to generate an acoustic signal to the subject, which acoustic signal provides a sound cue confirming a position for the estimated point of regard at a particular object of said objects in the surrounding environment, the sound cue simulating an acoustic source located at the estimated point of regard via three-dimensional positional audio effects.

13. The processing circuitry according to claim 10 , wherein the output interface is configured to generate the feedback signal such that an intensity of the at least one first sensory stimulus reflects a distance to a first surface element of an object intersecting a straight line between an eye-base line of the subject and the estimated point of regard, which first surface element is located closer to the eye-base line than any other surface element of objects in the surrounding environment along said straight line.

14. The processing circuitry according to claim 10 , wherein the reference point is located at a geometric center of a head mountable arrangement that incorporates an eyetracker configured to generate the gaze data, the geometric center representing an estimated point around which the head mountable arrangement is rotated when mounted on the subject.

15. The processing circuitry according to claim 10 , wherein the spatial data describes color and light intensity information reflected from objects in the surrounding environment, and the processing circuitry is configured to:
obtain at least one visual quality of at least one of said objects from the spatial data; and generate the feedback signal such that it reflects said at least one visual quality.

16. The processing circuitry according to claim 15 , wherein the at least one visual quality comprises at least one of a color, a surface texture and an object class describing said at least one object, and the output interface is configured to provide the feedback signal to cause at least one second sensory stimulus to be applied to the subject, which at least one second sensory stimulus reflects at least one of the color, the surface tex

Assignees

• Tobii Ab

Inventors

• Ratcliff Andrew
• Tornéus Daniel
• Lundberg Eli
• Andersson Henrik

Classifications

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G01B11/026

  by measuring distance between sensor and object (G01B11/0608 takes precedence) · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G02B27/017

  Head mounted · CPC title

• H04N13/344 Primary

  with head-mounted left-right displays · CPC title