Headwear with computer and optical element for use therewith and systems utilizing same

Patent metadata

Field	Value
Publication number	US-10509466-B1
Application number	US-201514853851-A
Country	US
Kind code	B1
Filing date	Sep 14, 2015
Priority date	May 11, 2011
Publication date	Dec 17, 2019
Grant date	Dec 17, 2019

Abstract

An apparatus for mounting on a head including a frame, A face-wearable near-ocular optics and a micro-display for displaying data in front of the eyes is provided. A computing device is coupled to the micro-display. At least one sensor is coupled to the computing device for receiving biometric human information.

First claim

What is claimed is:

1. An apparatus comprising:
a support structure for mounting on the head of a user;
a microcomputer carried by the support structure;
an inwardly-facing camera mounted on the support structure, the inwardly-facing camera producing eye tracking video of an eye that is processed by the microcomputer, wherein the microcomputer processes the eye tracking video to track pupil position;
an outwardly-facing camera mounted on the support structure, the outwardly-facing camera capturing images similar to images capturable by the left and right eyes of the user;
a display mounted on the support structure, wherein the microcomputer generates an image stream from the captured images of the outwardly-facing camera and causes the image stream to be displayed on the display as displayed data, wherein the microcomputer further alters the displayed data to cause a rendered object in the displayed data to move according to an eye movement of the user, based on the tracked pupil position; and
a transceiver coupled with the microcomputer, wherein the microcomputer executes a first part of a procedure to extract data from images captured by the outwardly-facing camera, and sends a second part of the procedure to a mobile computer via the transceiver for execution of the second part of the procedure on the mobile computer, the result of the second part of the procedure made available to the microcomputer.

2. The apparatus of claim 1 wherein the microcomputer processes the eye tracking video to detect eye corners using oriented gradient features and averages the positions of the eye corners to obtain an x,y location for the eye.

3. The apparatus of claim 1 ,
wherein the microcomputer processes the eye tracking video to calculate a two-dimensional displacement between the center of the eye and the center of the pupil.

4. The apparatus of claim 3 wherein the microcomputer converts the two-dimensional displacement to a three-dimensional gaze vector for the eye.

5. The apparatus of claim 4 wherein the microcomputer computes a three-dimensional point in space where two gaze rays intersect to establish a z coordinate vergence point.

6. The apparatus of claim 4 where the microcomputer applies an affine transformation in three-dimensions to convert the two-dimensional displacement into the three-dimensional gaze vector for the eye.

7. The apparatus of claim 3 wherein the pupil position is used to alter a position of data projected on a display mounted on the support structure.

8. The apparatus of claim 1 , wherein the microcomputer processes the images captured by the outwardly-facing camera by overlaying virtual objects onto the images.

9. An apparatus, comprising:
a support structure for mounting on the head of a user:
a microcomputer carried by the support structure;
an inwardly-facing camera mounted on the support structure, the inwardly-facing camera producing eye tracking video of an eye that is processed by the microcomputer, wherein the microcomputer processes the eye tracking video to track pupil position;
a sensor fusion controller to combine data from a plurality of sensors, wherein the sensor fusion controller combines the data from the plurality of sensors using a Kalman filter;
an outwardly-facing camera mounted on the support structure, the outwardly-facing camera capturing images similar to images capturable by the left and right eyes of the user;
a display mounted on the support structure, wherein the microcomputer generates an image stream from the captured images of the outwardly-facing camera and causes the image stream to be displayed on the display as displayed data, wherein the microcomputer further alters the displayed data to cause a rendered object in the displayed data to move according to an eye movement of the user, based on the tracked pupil position; and
a transceiver coupled with the microcomputer, wherein the microcomputer executes a first part of a procedure to extract data from images captured by the outwardly-facing camera, and sends a second part of the procedure to a mobile computer via the transceiver for execution of the second part of the procedure on the mobile computer, the result of the second part of the procedure made available to the microcomputer.

10. The apparatus of claim 9 wherein the microcomputer performs a calibration operation by projecting a sequence of objects on the display and collecting viewed object acknowledgements from a user.

11. The apparatus of claim 9 where the plurality of sensors comprise at least one of a group comprising: an accelerometer, a gyroscope, a magnetometer, an altimeter, a clock, and a thermometer.

12. A method comprising:
producing, using an inwardly-facing camera of an apparatus for mounting on the head of a user, an eye tracking video of an eye;
processing, h a microcomputer of the apparatus, the eye tracking video to track pupil position;
capturing, using an outwardly-facing camera of the apparatus, images similar to images captured by the left and right eyes of the user;
generating, by the microcomputer of the apparatus, an image stream from the captured images of the outwardly-facing camera;
causing, by the microcomputer of the apparatus, the image stream to be displayed on a display of the apparatus as displayed data;
altering, by the microcomputer of the apparatus, the displayed data to cause a rendered object of the displayed data to move according to an eye movement of the user, based on the tracked pupil position;
executing, by the microcomputer of the apparatus, a first part of a procedure to extract data from images captured by the outwardly-facing camera of the apparatus;
determining, by the microcomputer of the apparatus, a mobile computer to execute a second part of the procedure;
transmitting, by the microcomputer of the apparatus, the second part of the procedure to the mobile computer for execution of the second part of the procedure OD the mobile computer; and
receiving, by the microcomputer of the apparatus, a result of the execution of the second part of the procedure from the mobile computer.

13. The method of claim 12 , further comprising:
sending the displayed data to a networked computer via a communication network.

14. The method of claim 12 , wherein the result of the procedure is made available to the mobile computer.

Assignees

• Snap Inc

Inventors

• Miller Erick
• Rodriguez Jonathan

Classifications

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B2027/0116

  comprising devices for correcting chromatic aberration · CPC title

• G02B27/0176

  characterised by mechanical features · CPC title

• G02B3/005

  arranged along a single direction only, e.g. lenticular sheets (G02B3/0043 takes precedence) · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title