Controlling focal parameters of a head mounted display based on estimated user age

Patent metadata

Field	Value
Publication number	US-10706281-B2
Application number	US-201816101316-A
Country	US
Kind code	B2
Filing date	Aug 10, 2018
Priority date	Aug 10, 2018
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

The disclosure herein describes controlling focal parameters of a head mounted display based on an estimated user age to account for increasing likelihood of presbyopia as user age increases. The head mounted display uses eye tracking sensors to collect ocular metric data associated with ocular features of a user's eye and a user age estimate is calculated based on analysis of the ocular metric data using a machine learning algorithm and an ocular metric data set. Further, a confidence value of the user age estimate is calculated based on the analysis of the ocular metric data. Then, focal parameters of the visual display of the head mounted display are controlled based on the user age estimate and confidence value. The described focal control method provides a seamless, automated way for a head mounted display system to adjust settings to provide a sharp, in-focus user experience for users of all ages.

First claim

What is claimed is:

1. A system for controlling focal parameters of a visual display based on ocular features, the system comprising:
at least one processor;
a visual display including at least one eye tracking sensor; and
at least one memory communicatively coupled to the at least one processor and comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the at least one processor to:
collect, via the at least one eye tracking sensor, ocular metric data associated with at least one ocular feature of a user's eye;
calculate a user age estimate based on analysis of the ocular metric data using at least one machine learning algorithm and an associated ocular metric data set;
calculate a confidence value associated with the user age estimate based on the analysis of the ocular metric data; and
control at least one focal parameter of the visual display based on the calculated user age estimate and the associated confidence value.

2. The system of claim 1 , wherein the analysis of the ocular metric data includes comparing the collected ocular metric data to age range data from the ocular metric data set to determine an age range estimate.

3. The system of claim 2 , wherein analysis of the ocular metric data further includes:
calculating a plurality of age estimate values, wherein each age estimate value is based on an ocular metric data value of the collected ocular metric data;
applying a defined metric weight associated with each ocular metric data value to the associated age estimate value to form a plurality of weighted age estimate values; and
combining the plurality of weighted age estimate values to generate the user age estimate.

4. The system of claim 1 , wherein the ocular metric data includes at least one or pupil size data, pupil accommodation data, pupil shape data, blink dynamic data, saccade dynamic data, eye lid characteristic data, and progressive lens data.

5. The system of claim 1 , wherein controlling the at least one focal parameter includes at least one of fixing a focal plane of display output to greater than or equal to 1 meter, applying optical corrections, and enabling variable and multi-focus capabilities.

6. The system of claim 1 , wherein controlling the at least one focal parameter includes at least one of prompting a user to confirm a suggested focal parameter modification and automatically applying a focal parameter modification.

7. The system of claim 1 , the at least one memory and the computer program code configured to, with the at least one processor, further cause the at least one processor to:
collect feedback based on controlling the at least on focal parameter; and
update the machine learning algorithm based on the collected feedback.

8. The system of claim 7 , wherein the collected feedback includes at least one of data based on user behavior after controlling the at least one focal parameter, data indicating a user manually changing the at least one controlled focal parameter, response data from prompting the user about the clarity of output of the visual display, and response data from prompting the user about the user's current age.

9. The system of claim 7 , wherein updating the machine learning algorithm includes adjusting metric weights associated with ocular metric values and applied to age estimate values.

10. A computerized method for controlling focal parameters of a visual display based on ocular features, the method comprising:
collecting, by a processor using at least one eye tracking sensor, ocular metric data associated with at least one ocular feature of a user's eye;
calculating, by the processor, a user age estimate based on analysis of the ocular metric data using at least one machine learning algorithm and an associated ocular metric data set;
calculating, by the processor, a confidence value associated with the user age estimate based on the analysis of the ocular metric data; and
controlling, by the processor, at least one focal parameter of the visual display based on the calculated user age estimate and the associated confidence value.

11. The computerized method of claim 10 , wherein the analysis of the ocular metric data includes comparing the collected ocular metric data to age range data from the ocular metric data set to determine an age range estimate.

12. The computerized method of claim 11 , wherein analysis of the ocular metric data further includes:
calculating a plurality of age estimate values, wherein each age estimate value is based on an ocular metric data value of the collected ocular metric data;
applying a defined metric weight associated with each ocular metric data value to the associated age estimate value to form a plurality of weighted age estimate values; and
combining the plurality of weighted age estimate values to generate the user age estimate.

13. The computerized method of claim 10 , wherein the ocular metric data includes at least one or pupil size data, pupil accommodation data, pupil shape data, blink dynamic data, saccade dynamic data, eye lid characteristic data, and progressive lens data.

14. The computerized method of claim 10 , wherein controlling the at least one focal parameter includes at least one of fixing a focal plane of display output to greater than or equal to 1 meter, applying optical corrections, and enabling variable and multi-focus capabilities.

15. The computerized method of claim 10 , wherein controlling the at least one focal parameter includes at least one of prompting a user to confirm a suggested focal parameter modification and automatically applying a focal parameter modification.

16. The computerized method of claim 10 , further comprising:
collecting feedback based on controlling the at least on focal parameter; and
updating the machine learning algorithm based on the collected feedback.

17. One or more computer storage media having computer-executable instructions for controlling focal parameters of a visual display based on ocular features that, upon execution by a processor, cause the processor to at least:
collect, via at least one eye tracking sensor of the visual display, ocular metric data associated with at least one ocular feature of a user's eye;
calculate a user age estimate based on analysis of the ocular metric data using at least one machine learning algorithm and an associated ocular metric data set;
calculate a confidence value associated with the user age estimate based on the analysis of the ocular metric data; and
control at least one focal parameter of the visual display based on the calculated user age estimate and the associated confidence value.

18. The one or more computer storage media of claim 17 , wherein the analysis of the ocular metric data includes comparing the collected ocular metric data to age range data from the ocular metric data set to determine an age range estimate.

19. The one or more computer storage media of claim 18 , wherein analysis of the ocular metric data further includes:
calculating a plurality of age estimate values, wherein each age estimate value is based on an ocular metric data value of the collected ocular metric data;
applying a defined metric weight associated with each ocular metric data value to the associated age estimate value to form a plurality of weighted age estimate values; and
combining the plurality of weighted age estimate values to generate the user age estimate.

20. The one or more computer storage media of

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Held Robert Thomas
• Robinson Richard Zachary
• Aholt Christopher Charles

Classifications

• G06V40/178

  estimating age from face image; using age information for improving recognition · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title