Imaging modification, display and visualization using augmented and virtual reality eyewear

Patent metadata

Field	Value
Publication number	US-11327312-B2
Application number	US-202016790576-A
Country	US
Kind code	B2
Filing date	Feb 13, 2020
Priority date	Jul 25, 2016
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

A display system can include a head-mounted display configured to project light to an eye of a user to display augmented reality image content to the user. The display system can include one or more user sensors configured to sense the user and can include one or more environmental sensors configured to sense surroundings of the user. The display system can also include processing electronics in communication with the display, the one or more user sensors, and the one or more environmental sensors. The processing electronics can be configured to sense a situation involving user focus, determine user intent for the situation, and alter user perception of a real or virtual object within the vision field of the user based at least in part on the user intent and/or sensed situation involving user focus. The processing electronics can be configured to at least one of enhance or de-emphasize the user perception of the real or virtual object within the vision field of the user.

First claim

What is claimed is:

1. A head-mounted display system configured to project light to an eye of a user to display augmented reality image content in a vision field of said user, the vision field having a central region and a peripheral region disposed about said central region, said head-mounted display system comprising:
a frame configured to be supported on a head of the user;
a head-mounted display disposed on the frame, said display configured to project light into said user's eye to display augmented reality image content to the user's vision field, at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears said head-mounted display such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user and said head-mounted display;
one or more user sensors configured to sense the user;
one or more environmental sensors configured to sense surroundings of the user;
processing electronics in communication with the display, the one or more user sensors, and the one or more environmental sensors, the processing electronics configured to:
sense a situation involving increased user focus; and

based at least in part on sensing the increased focus on a real or virtual peripheral object in the peripheral region of the user's vision field, de-emphasize image content in a portion of the vision field other than the peripheral object, wherein sensing the increased user focus on the peripheral object is based at least in part on the user's eye squinting at the peripheral object or fluctuations in accommodation of the user's eye when directed at the peripheral object.

2. The display system of claim 1 , wherein the processing electronics are configured to de-emphasize the image content in the portion of the vision field other than the peripheral object by altering at least one of contrast, opacity, color, color saturation, color balance, size, background, brightness, edges, or sharpness of the image content in the portion of the vision field other than the peripheral object.

3. The display system of claim 1 , wherein the one or more environmental sensors comprise a depth sensor, a pair of binocular world cameras, a geolocation sensor, a proximity sensor, or a GPS.

4. The display system of claim 1 , wherein the one or more user sensors comprise one or more cameras.

5. The display system of claim 1 , wherein the processing electronics are further configured to determine the user intent for the situation and alter the user perception of the real or virtual peripheral object within the vision field of the user based at least in part on sensing the increased focus.

6. The display system of claim 1 , wherein said situation involving increased user focus comprises driving a motor vehicle.

7. The display system of claim 1 , wherein said one or more environmental sensors comprise a sensor configured to detect a radio signal.

8. The display system of claim 1 , wherein said one or more environmental sensors comprise a sensor configured to detect a blue tooth signal from an automobile.

9. The display system of claim 6 , wherein said processing electronics are configured to de-emphasize image content in the portion of the vision field other than the peripheral object based at least in part on one or more data records regarding the user, said one or more data records comprising a driving record of said user.

10. A head-mounted display system configured to project light to an eye of a user to display augmented reality image content, said user's eye having a vision field having a central region and a peripheral region disposed about said central region, said head-mounted display system comprising:
a frame configured to be supported on a head of the user;
a head-mounted display disposed on the frame, said display configured to project light into said user's eye so as to present image content at said central region of said user's vision field, at least a portion of said display being transparent and disposed at a location in front of the user's eye when the user wears said head-mounted display such that said transparent portion transmits light from a portion of the environment in front of the user and said head-mounted display to the user's eye to provide a view of said portion of the environment in front of the user and said head-mounted display;
processing electronics in communication with said display to control presentation of image content on said display, the processing electronics configured to:
sense a situation involving increased user focus; and
based at least in part on sensing the increased focus on a real or virtual peripheral object in the peripheral region of the user's vision field, de-emphasize image content in a portion of the vision field other than the peripheral object, wherein sensing the increased user focus on the peripheral object is based at least in part on the number of occasions the user's eye is directed at the peripheral object, the amount of time the user's eye is directed at the peripheral object, the user's eye squinting at the peripheral object, or measurement of accommodation of the user's eye when directed at the peripheral object,
wherein the system is configured to provide an alert to the user to indicate the de-emphasized image content.

11. The system of claim 10 , further comprising one or more sensors configured to monitor the environment.

12. The system of claim 11 , wherein said one or more sensors comprise one or more outward-facing image capture devices configured to image said environment.

13. The system of claim 11 , wherein said one or more sensors comprise a distance measuring device.

14. The system of claim 10 , further comprising an eye tracking device configured to track position and/or movement of said user's eye.

15. The system of claim 10 , wherein the head-mounted display system is configured to process image content presented to at least a portion of said central region of the user's vision field differently in comparison to image content presented to the peripheral region of the user's vision field.

16. The system of claim 10 , wherein the head-mounted display system is configured to de-emphasize image content in the portion of the vision field other than the peripheral object by reducing the size of image content in the portion of the vision field other than the peripheral object.

17. The system of claim 10 , wherein the head-mounted display system is configured to de-emphasize image content in the portion of the vision field other than of the peripheral object by decreasing brightness, visibility, sharpness, or contrast in image content in the portion of the vision field other than the peripheral object.

18. The system of claim 1 , wherein the processing electronics are configured to de-emphasize the image content in the portion of the vision field other than the peripheral object by reducing the size of image content in the portion of the vision field other than the peripheral object.

19. The system of claim 1 , wherein the processing electronics are configured to de-emphasize the image content in the portion of the vision field other than the peripheral object by decreasing brightness, visibility, sharpness, or contrast of image content in the portion of the vision field other than of the peripheral object.

20. The system of claim 1 , wherein the head-mounted dis

Assignees

• Magic Leap Inc

Inventors

• Robaina Nastasja U
• Samec Nicole Elizabeth
• Harrises Christopher M
• Abovitz Rony
• Baerenrodt Mark
• Schmidt Brian Lloyd

Classifications

• G16H20/40 Primary

  relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture · CPC title

• G02B27/0172 Primary

  characterised by optical features · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title