Method and device for switching input modalities of a displaying device

Patent metadata

Field	Value
Publication number	US-10983359-B2
Application number	US-201916711056-A
Country	US
Kind code	B2
Filing date	Dec 11, 2019
Priority date	Dec 11, 2018
Publication date	Apr 20, 2021
Grant date	Apr 20, 2021

Abstract

Disclosed is a method for switching user input modality of a displaying device displaying an interactable region. The displaying device is in communication with a first input modality and a second input modality. The first input modality is an eye tracker configured to determine gaze data of a user and the second input modality is an input modality other than an eye tracker configured to determine a pointing ray. The first input modality is selected as the input modality of the displaying device. The method comprises determining whether the pointing ray of the second input modality is intersecting with the interactable region. The method further comprises, based on the determining switching the input modality of the displaying device to the second input modality when the pointing ray of the second input modality is intersecting with the interactable region.

First claim

The invention claimed is:

1. A method for switching user input modality of a displaying device displaying an interactable region, wherein the displaying device is in communication with a first input modality and a second input modality wherein the first input modality is an eye tracker configured to determine gaze data of a user and the second input modality is an input modality other than an eye tracker configured to determine a pointing ray wherein the first input modality is selected as the input modality of the displaying device the method comprising:
determining whether the pointing ray of the second input modality is intersecting with the interactable region;
based on the determining switching the input modality of the displaying device to the second input modality when the pointing ray of the second input modality is intersecting with the interactable region, and
wherein the gaze data of the user comprises a gaze ray; and
wherein the switching is only performed when an angle between the gaze ray and the pointing ray is below a predetermined threshold.

2. The method according to claim 1 , wherein the displaying device is a physical display wherein the interactable region is displayed at the physical display.

3. The method according to claim 1 , wherein the displaying device is a HMD display and the interactable region is projected at a predetermined distance by the HMD display.

4. The method according to claim 1 , wherein the displaying device is an HMD display and the interactable region is defined by predetermined maximum angle between a direction of the head of the user of the HMD and the pointing ray of the second input modality.

5. The method according to claim 1 , wherein the switching is only performed if the pointing ray of the second input modality is also intersecting with a user-selectable region in the interactable region.

6. The method according to claim 1 , further comprising:
determining a normal to the interactable region and further wherein the switching is performed in an angle between a normal to the interactable region and the pointing ray of the second input modality is below a predetermined threshold.

7. The method according to claim 1 , further comprising:
prior to the switching determining a direction of the head of the user of the displaying device, and further wherein the switching is only performed if an angle between the direction of the head of the user and the pointing ray of the second input modality is below a predetermined threshold.

8. The method according to claim 1 , wherein the displaying device is in communication with a third input modality, the third input modality being an input modality other than an eye-tracker which is configured to determine a pointing ray, wherein the method further comprises:
determining whether the pointing ray of the third input modality is intersecting with the interactable region; and
based on the determining switching input modality to the third input modality if the pointing ray of the third input modality is intersecting with the interactable region.

9. The method according to claim 1 , wherein at least one of the second and a third input modalities comprises a pointing device, such as a hand-held pointer or a hand-gesture based pointing device.

10. The method according to claim 1 , wherein a direction of the pointing ray of an input modality is indicated by the input modality.

11. A displaying device adapted to display an interactable region, wherein the displaying device is in communication with a first input modality and a second input modality wherein the first input modality is an eye tracker configured to determine gaze data of a user and the second input modality is an input modality other than an eye tracker configured to determine a pointing ray wherein the first input modality is selected as the input modality of the displaying device, the displaying device comprising:
processing circuitry; and
a memory, said memory containing instructions executable by said processing circuitry, whereby said displaying device is operative for:
determining whether the pointing ray of the second input modality is intersecting with the interactable region;
based on the determining, switching the input modality of the displaying device to the second input modality when the pointing ray of the second input modality is intersecting with the interactable region, and
wherein the gaze data of the user comprises a gaze ray; and
wherein the switching is only performed when an angle between the gaze ray and the pointing ray is below a predetermined threshold.

12. A non-transitory computer-readable storage medium storing instructions that, upon execution by a display device causes the displaying device to perform the following steps:
determining whether the pointing ray of the second input modality is intersecting with the interactable region;
based on the determining, switching the input modality of the displaying device to the second input modality when the pointing ray of the second input modality is intersecting with the interactable region, and
wherein the gaze data of the user comprises a gaze ray; and
wherein the switching is only performed when an angle between the gaze ray and the pointing ray is below a predetermined threshold.

13. A carrier containing the computer program according to claim 12 , wherein the carrier is one of an electronic signal, optical signal, radio signal or computer readable storage medium.

Assignees

• Tobii Ab

Inventors

• Blomqvist Niklas
• Thunström Robin
• Rådell Dennis
• Biedert Ralf

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F2203/0331

  Finger worn pointing device · CPC title

• G06F3/033

  Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title