Techniques for setting focus in mixed reality applications

Patent metadata

Field	Value
Publication number	US-10044925-B2
Application number	US-201615240850-A
Country	US
Kind code	B2
Filing date	Aug 18, 2016
Priority date	Aug 18, 2016
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

Described are examples for capturing one or more real world images for display with one or more computer-generated images (e.g., holograms). One or more computer-generated images for overlaying over one or more real world images can be received, and a depth for overlaying at least one of the computer-generated images over the one or more real world images can be determined. A lens of a camera can be focused based on this depth, and the one or more real world images can be captured via the camera with the lens focused based on the depth. The one or more real world images can be provided as one or more mixed reality images with the one or more computer-generated images overlaid on the one or more real world images, such that the image is focused on objects near the one or more computer-generated images.

First claim

What is claimed is:

1. A computing device, comprising:
a camera comprising a lens configured to capture and provide one or more real world images; and
at least one processor configured to determine whether to transform the one or more real world images to a mixed reality image;
wherein when the at least one processor determines not to transform the one or more real world images, the at least one processor is configured to perform one or more auto-focusing processes of the camera to focus the one or more real world images; and
wherein when the at least one processor determines to transform the one or more real world images, the at least one processor is configured to:
overlay one or more computer-generated images over the one or more real world images; and
override, based on determining to transform the one or more real world images, the one or more auto-focusing processes of the camera by causing the camera to focus the lens for capturing the one or more real world images based at least in part on a depth at which the at least one processor overlays at least one of the one or more computer-generated images.

2. The computing device of claim 1 , wherein the at least one processor is further configured to cause the camera to focus the lens at the depth of the at least one of the one or more computer-generated images.

3. The computing device of claim 1 , wherein the at least one processor is configured to determine a selection of the at least one computer-generated image based on a size of the at least one computer-generated image, and wherein the at least one processor is configured to cause the camera to focus the lens at least in part in response to the selection.

4. The computing device of claim 1 , wherein the at least one processor is configured to cause the camera to focus the lens at the depth corresponding to a point of the at least one of the one or more computer-generated images having a level of detail that achieves a threshold.

5. The computing device of claim 1 , wherein the at least one processor is configured to cause the camera to focus the lens at the depth corresponding to a size or distance from the computing device of the at least one of the one or more computer-generated images.

6. The computing device of claim 1 , wherein the at least one processor is configured to receive a selection of the at least one of the one or more computer-generated images, and wherein the at least one processor is configured to cause the camera to focus the lens at least in part in response to the selection.

7. The computing device of claim 6 , wherein the selection corresponds to a detected gesture to select the at least one of the one or more computer-generated images presented on an interface on a display.

8. The computing device of claim 6 , further comprising a gaze tracking sensor configured to track a position of one or more eyes of a user, wherein the selection corresponds to a determination that the one or more eyes of the user are positioned to view the at least one of the one or more computer-generated images.

9. The computing device of claim 6 , wherein the selection corresponds to determining one or more heuristics of the at least one of the one or more computer-generated images.

10. The computing device of claim 1 , further comprising a display for displaying the one or more real world images overlaid with the one or more computer-generated images, wherein the display is at least one of positioned on the computing device or remotely located from the computing device.

11. A method for capturing one or more real world images for providing with one or more computer-generated images, comprising:
receiving one or more computer-generated images for overlaying over one or more real world images;
determining a depth for overlaying at least one of the one or more computer-generated images over the one or more real world images;
overriding, based on receiving the one or more computer-generated images, an auto-focusing process of a camera at least in part by instead focusing a lens of a camera based on the depth;
capturing the one or more real world images via the camera with the lens focused based on the depth; and
providing the one or more real world images as one or more mixed reality images with the one or more computer-generated images overlaid on the one or more real world images.

12. The method of claim 11 , wherein focusing the lens further comprises focusing the lens at the depth for overlaying the at least one of the one or more computer-generated images.

13. The method of claim 11 , further comprising determining a selection of the at least one computer-generate image based on a size of the at least one computer-generated image, and wherein focusing the lens is based at least in part on the selection.

14. The method of claim 11 , wherein focusing the lens further comprises focusing the lens at the depth corresponding to a point of the at least one of the one or more computer-generated images having a level of detail that achieves a threshold.

15. The method of claim 11 , further comprising receiving a selection of the at least one of the one or more computer-generated images, and wherein focusing the lens is based at least in part on the selection.

16. The method of claim 15 , further comprising detecting a gesture to select the at least one of the one or more computer-generated images presented on an interface on the display, wherein the selection corresponds to the gesture.

17. The method of claim 15 , further comprising:
tracking position of one or more eyes of a user; and
determining that the one or more eyes of the user are positioned to view the at least one of the one or more computer-generated images, wherein the selection corresponds to the determination.

18. A non-transitory computer-readable medium comprising code for capturing one or more real world images for providing with one or more computer-generated images, the code comprising:
code for receiving one or more computer-generated images for overlaying on one or more real world images;
code for determining a depth for overlaying at least one of the one or more computer-generated images on the one or more real world images;
code for overriding, based on receiving the one or more computer-generate images, an auto-focusing process of a camera at least in part by instead focusing a lens of a camera based on the depth;
code for capturing the one or more real world images via the camera with the lens focused based on the depth; and
code for providing the one or more real world images as one or more mixed reality images with the one or more computer-generated images overlaid on the one or more real world images.

19. The non-transitory computer-readable medium of claim 18 , wherein the code for focusing the lens focuses the lens at the depth for overlaying the at least one of the one or more computer-generated images.

20. The non-transitory computer-readable medium of claim 18 , wherein the code further comprises code for determining a selection of the at least one computer-generate image based on a size of the at least one computer-generated image, and wherein focusing the lens is based at least in part on the selection.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Li Hang
• Jain Vishal
• Lei Maria Cheng
• GUYMAN William Douglas

Classifications

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• H04N23/959 Primary

  by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title