AR glasses with predictive control of external device based on event input

Patent metadata

Field	Value
Publication number	US-10860100-B2
Application number	US-201715433757-A
Country	US
Kind code	B2
Filing date	Feb 15, 2017
Priority date	Feb 28, 2010
Publication date	Dec 8, 2020
Grant date	Dec 8, 2020

Abstract

This disclosure concerns an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment and the displayed content, wherein the eyepiece includes predictive control of external device based on an event input.

First claim

The invention claimed is:

1. A system, comprising:
a head-mounted display device including an optical assembly through which at least a part of a surrounding environment and also displayed content are viewable, a camera configured to image the surrounding environment, and an image source configured to introduce content for display via the optical assembly; and
a peripheral hand motion sensing device configured to provide input to the head-mounted display device, the peripheral hand motion sensing device comprising one or more motion sensors configured to sense and send motion data to the head-mounted display device, the head-mounted display device being configured to use motion data from the peripheral hand motion sensing device for one or more control functions of the head-mounted display device when a hand is not in a field of view of the camera, wherein the head-mounted display device is configured to detect a motion of the hand coming into the field of view of the camera from out of the field of view of the camera, in response disable a control function of the peripheral hand motion sensing device, and after disabling, detect a control input to the head-mounted display device based on motion of the hand in the field of view of the camera.

2. The system of claim 1 , wherein the head-mounted display device is configured to enable the control function of the peripheral hand motion sensing device when the hand is no longer detected in a field of view of the camera of the head-mounted display device.

3. The system of claim 1 , wherein the peripheral hand motion sensing device is configured to sense motion of a finger and send motion data regarding the motion of the finger to the head-mounted display device.

4. The system of claim 1 , wherein the peripheral hand motion sensing device is configured to receive input to control a cursor displayed by the head-mounted display device.

5. The system of claim 4 , wherein the head-mounted display device is configured to enable control of the cursor when the motion of the hand is detected within the field of view of the camera of the head-mounted display device.

6. The system of claim 4 , wherein the head-mounted display device is configured to disable the hand motion sensing device from controlling the cursor when the motion of the hand is detected within the field of view of the camera of the head-mounted display device.

7. The system of claim 1 , wherein the peripheral hand motion sensing device comprises one or more of a gyroscope, an accelerometer, a velocity sensor, a force sensor, a pressure sensor, an optical sensor, and a proximity sensor.

8. The system of claim 1 , wherein the head-mounted display device comprises an optical transmitter configured to emit signals detectable by the peripheral hand motion sensing device, and wherein the peripheral hand motion sensing device is configured to track hand motions based upon the signals emitted by the optical transmitter.

9. The system of claim 1 , wherein the peripheral hand motion sensing device is configured to track hand gestures within the field of view of the camera of the head-mounted display device and provide control instructions to the head-mounted display device based upon the hand gestures.

10. The system of claim 1 , wherein the peripheral hand motion sensing device comprises a user-actuatable input device configured to receive input to control a user interface of the head-mounted display device.

11. A system, comprising:
a head-mounted display device including an optical assembly through which at least a part of a surrounding environment and also displayed content are viewable, a camera configured to image the surrounding environment, and an image source configured to introduce content for display via the optical assembly;
a peripheral hand motion sensing device comprising one or more motion sensors configured to sense hand motion data as input to control a user interface of the head-mounted display device, the head-mounted display device being configured to use hand motion data from the peripheral hand motion sensing device for one or more control functions of the head-mounted display device when a hand is not in a field of view of the camera;
a processor; and
a storage device comprising instructions executable by the processor to
detect a motion of the hand coming into the field of view of the camera of the head-mounted display device from out of the field of view of the camera,
in response, disable a control function of the peripheral hand motion sensing device,
after disabling, detect a control input to the head-mounted display device based on motion of the hand in the field of view of the camera, and
enable the control function of the peripheral hand motion sensing device when the hand is no longer detected in the field of view of the camera.

12. The system of claim 11 , wherein the instructions are executable to disable control of a cursor by the peripheral hand motion sensing device when the motion of a hand is detected in the field of view of the camera of the head-mounted display device.

13. The system of claim 11 , wherein the motion of the hand is a first motion, and wherein the instructions are further executable to detect a second motion of the hand in the field of view of the camera of the head-mounted display device, and in response, enable the control function of the peripheral hand motion sensing device.

14. The system of claim 11 , wherein the peripheral hand motion sensing device is configured to sense motion data of a finger and send the motion data of the finger to the head-mounted display device.

15. The system of claim 11 , wherein the peripheral hand motion sensing device comprises one or more of a gyroscope, an accelerometer, a velocity sensor, a force sensor, a pressure sensor, an optical sensor, and a proximity sensor.

16. The system of claim 11 , wherein the head-mounted display device comprises an optical transmitter configured to emit signals detectable by the peripheral hand motion sensing device, and wherein the peripheral hand motion sensing device is configured to track hand motions based upon the signals emitted by the optical transmitter.

17. The system of claim 11 , wherein the peripheral hand motion sensing device is configured to track hand gestures within the field of view of the camera of the head-mounted display device and provide control instructions to the head-mounted display device based upon the hand gestures.

18. On a system comprising a head-mounted display device including an optical assembly through which at least a part of a surrounding environment and also displayed content are viewable, a camera configured to image the surrounding environment, and an image source configured to introduce content for display via the optical assembly, and the system also comprising a peripheral hand motion sensing device comprising one or more motion sensors, a method comprising:
using motion data from the peripheral hand motion sensing device for one or more control functions of the head-mounted display device when a hand is not in a field of view of the camera,
detecting a motion of the hand coming into the field of view of the camera of the head-mounted display device from out of the field of view of the camera,
in response, disabling a control function of the peripheral hand motion sensing device,
after disabling, detecting a control input to the head-mounted display device based on motion of the hand in the field of view of the camera, and
enabling the control function of the peripheral hand motion sensing devic

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Osterhout Ralph F
• Haddick John D
• Lohse Robert Michael

Classifications

• G06F3/0488 Primary

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/014 Primary

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G06F13/00

  Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units (interface circuits for specific input/output devices G06F3/00 {; multiprogram control therefor  G06F9/46}; multiprocessor systems  G06F15/16 ) · CPC title

• G06F3/0227

  Cooperation and interconnection of the input arrangement with other functional units of a computer (G06F3/023 - G06F3/037 take precedence) · CPC title