Systems, devices, and methods for interacting with content displayed on head-mounted displays

Patent metadata

Field	Value
Publication number	US-10656822-B2
Application number	US-201715799642-A
Country	US
Kind code	B2
Filing date	Oct 31, 2017
Priority date	Oct 1, 2015
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

Systems, devices, and methods that enable sophisticated and inconspicuous interactions with content displayed on a head-mounted display are described. A head-mounted display includes an eye-tracker and the user also carries/wears a wireless portable interface device elsewhere on their body, such as a ring. The wireless nature of the portable interface device enables a small and unobtrusive form factor. The portable interface device includes an actuator that, when activated by the user, causes the portable interface device to wirelessly transmit a signal (e.g., a radio frequency signal or a sonic signal). A selection operation performed by the user is defined as the user activating the actuator of the portable interface device while the user is substantially concurrently gazing at a displayed object (as detected by the eye-tracker). In response to the selection operation, the head-mounted display displays a visual effect to the user.

First claim

The invention claimed is:

1. A method of operating a system, wherein the system comprises a head-mounted display, an eye-tracker carried by the head-mounted display, and an electronic ring with a form factor to be worn on a finger of a user, the electronic ring physically separate from the head-mounted display and the electronic ring including an actuator positioned and oriented to be activatable by a thumb of the user when the electronic ring is worn on the finger of the user, the method comprising:
displaying multiple objects within a field of view of at least one eye of the user by at least one display of the head-mounted display;
receiving a selection operation from the user by the system, wherein receiving a selection operation from the user by the system includes:
receiving, by the electronic ring, an activation from the user of the actuator of the electronic ring;
in response to receiving, by the electronic ring, the activation from the user of the actuator of the electronic ring, wirelessly transmitting a signal by the electronic ring;
wirelessly receiving the signal by a wireless receiver of the head-mounted display;
in response to wirelessly receiving the signal by a wireless receiver of the head-mounted display, requesting, by a processor, current gaze-direction data from the eye-tracker carried by the head mounted display; and
identifying, by the processor, a particular object at which the user is gazing among the multiple objects displayed by at least one display of the head-mounted display based on the current gaze-direction data from the eye-tracker; and

in response to receiving the selection operation from the user by the system:
displaying a visual effect by the at least one display of the head-mounted display based on the particular object,
wherein:
the electronic ring is batteryless and includes a mechanical resonator physically coupled to the actuator,
receiving, by the electronic ring, an activation from the user of the actuator of the electronic ring includes receiving, by the electronic ring, a mechanical actuation of the mechanical resonator from the user,
in response to the mechanical actuation, the mechanical resonator generates a sonic signal,
wirelessly transmitting a signal by the electronic ring includes wirelessly transmitting the sonic signal by mechanical resonator of the electronic ring, and
wirelessly receiving the signal by a wireless receiver of the head-mounted display includes wirelessly receiving the sonic signal by the wireless receiver of the head-mounted display, and wherein the wireless receiver of the head-mounted display includes at least one of: a microphone and/or a piezoelectric element tuned to be responsive to the sonic signal.

2. The method of claim 1 wherein the head-mounted display includes a processor and a non-transitory processor-readable storage medium communicatively coupled to the processor, the processor communicatively coupled to the eye-tracker, and wherein the non-transitory processor-readable storage medium stores processor-executable instructions and/or data and the method comprises executing, by the processor, the processor-executable instructions and/or data stored in the non-transitory processor-readable storage medium to cause:
the at least one display of the head-mounted display to display the multiple objects within the field of view of at least one eye of the user;
the wireless receiver of the head-mounted display to receive the signal transmitted by the electronic ring;
the processor to, in response to wirelessly receiving the signal by a wireless receiver of the head-mounted display, request current gaze-direction data from the eye-tracker carried by the head mounted display; and
the processor to identify a particular object at which the user is gazing among the multiple objects displayed by at least one display of the head-mounted display based on the current gaze-direction data from the eye-tracker; and
the at least one display of the head-mounted display to display the visual effect based on the particular object in response to the system receiving the selection operation from the user.

3. The method of claim 1 wherein the sonic signal generated by the mechanical resonator of the electronic ring includes an ultrasonic signal.

4. The method of claim 1 wherein the particular object includes at least one object selected from a group consisting of: a menu item, a graphical button, a keyboard key, a notification, a file, a folder, and an alphanumeric character.

5. The method of claim 4 wherein the particular object is a menu item, and wherein:
identifying, by the processor, a particular object at which the user is gazing among the multiple objects displayed by at least one display of the head-mounted display based on the current gaze-direction data from the eye-tracker includes identifying, by the processor, the menu item.

6. The method of claim 1 wherein the actuator includes a button positioned and oriented on the electronic ring to be activatable by inconspicuous, substantially hands-free inputs from the user, and wherein:
receiving, by the electronic ring, an activation from the user of the actuator of the electronic ring includes receiving, by the electronic ring, an inconspicuous, substantially hands-free activation from the user of the actuator of the electronic ring.

7. The method of claim 1 wherein:
receiving, by the electronic ring, an activation from the user of the actuator of the electronic ring includes receiving, by the electronic ring, an activation from the thumb of the user of the actuator of the electronic ring; and
in response to receiving, by the electronic ring, the activation from the user of the actuator of the electronic ring, wirelessly transmitting a signal by the electronic ring includes wirelessly transmitting the signal by the electronic ring in response to receiving, by the electronic ring, the activation from the thumb of the user of the actuator of the electronic ring.

Assignees

• Thalmic Labs Inc
• North Inc

Inventors

• Bailey Matthew
• Alexander Stefan

Classifications

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06F3/0487 Primary

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/02

  Input arrangements using manually operated switches, e.g. using keyboards or dials · CPC title