Composite image associated with a head-mountable device

Patent metadata

Field	Value
Publication number	US-9607440-B1
Application number	US-201615258960-A
Country	US
Kind code	B1
Filing date	Sep 7, 2016
Priority date	Nov 6, 2013
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

In one aspect, an HMD is disclosed that provides a technique for generating a composite image representing the view of a wearer of the HMD. The HMD may include a display and a front-facing camera, and may be configured to perform certain functions. For instance, the HMD may be configured to make a determination that a trigger event occurred and responsively both generate a first image that is indicative of content displayed on the display, and cause the camera to capture a second image that is indicative of a real-world field-of-view associated with the HMD. Further, the HMD may be configured to generate a composite image that combines the generated first image and the captured second image.

First claim

The invention claimed is:

1. A method comprising:
displaying content on a transparent or semi-transparent display of a head-mountable device (HMD), wherein the displayed content is perceived by a wearer of the HMD as being superimposed over a real-world field-of-view seen through the transparent or semi-transparent display;
making, by the HMD, a determination that a trigger event occurred;
responsive to making the determination that the trigger event occurred, both: (i) generating, by the HMD, a first image of the displayed content, and (ii) causing, by the HMD, a camera to capture a second image of the real-world field-of-view associated with the HMD; and
generating, by the HMD, a composite image that combines the generated first image, the captured second image, and an effect image by overlaying the generated first image on the effect image and overlaying the effect image on the captured second image, wherein the effect image improves the accuracy of the generated composite image's representation of the wearer's view, and wherein the effect image is neither displayed on the display of the HMD nor part of the real-world field-of-view associated with the HMD.

2. The method of claim 1 , wherein the trigger event comprises at least one event from a group of events comprising: a gesture being made, a voice command being spoken, and a button being pressed.

3. The method of claim 1 , wherein responsive to making the determination that the trigger event occurred, (i) generating, by the HMD, the first image, and (ii) causing, by the HMD, the camera to capture the second image comprises responsive to making the determination, substantially simultaneously (i) generating, by the HMD, the first image, and (ii) causing, by the HMD, the camera to capture the second image.

4. The method of claim 3 , wherein causing, by the HMD, the camera to capture the second image causes the HMD to, proximate a time that the camera captures the second image, display a visual effect on the display for a duration.

5. The method of claim 1 , wherein the generated first image comprises a screenshot of content displayed on the display.

6. The method of claim 1 , wherein the generated first image comprises a modified version of a screenshot of content displayed on the display.

7. The method of claim 1 , wherein in the generated composite image, the generated first image is aligned with the captured second image such that the generated composite image is indicative of a view of a user wearing the HMD proximate a time that the trigger event occurred.

8. The method of claim 1 , wherein the effect image comprises a glass-texture effect.

9. The method of claim 1 , wherein the generating the composite image occurs in response to making the determination.

10. The method of claim 1 , further comprising:
responsive to making the determination that the trigger event occurred, determining, by the HMD, data relating to the HMD's operation proximate a time that the trigger event occurred, wherein the generated composite image further includes an indication of the determined data.

11. The method of claim 10 , wherein the data comprises audio data indicative of audio being played by the HMD proximate a time that the trigger event occurred.

12. The method of claim 10 , wherein the data comprises environmental data indicative of the environment in which the HMD is operating proximate a time that the trigger event occurred.

13. The method of claim 10 , wherein the data comprises diagnostic or state data related to the HMD proximate a time that the trigger event occurred.

14. The method of claim 1 , further comprising displaying on the display the generated composite image.

15. The method of claim 1 , wherein the display comprises a transparent display.

16. A head-mountable device (HMD) comprising:
a transparent or semi-transparent display;
a camera positioned for capturing a real-world field-of-view associated with the HMD;
a processor; and
a non-transitory computer-readable medium including program instructions that, when executed by the processor, cause performance of a set of functions comprising:
displaying content on the transparent or semi-transparent display, wherein the displayed content is perceived by a wearer of the HMD as being superimposed over the real-world field-of-view seen through the transparent or semi-transparent display;
making a determination that a trigger event occurred;
responsive to making the determination that the trigger event occurred, (i) generating a first image of the displayed content, and (ii) causing the camera to capture a second image of the real-world field-of-view; and
generating, by the HMD, a composite image that combines the generated first image, the captured second image, and an effect image by overlaying the generated first image on the effect image and overlaying the effect image on the captured second image, wherein the effect image improves the accuracy of the generated composite image's representation of the wearer's view, and wherein the effect image is neither displayed on the display of the HMD nor part of the real-world field-of-view associated with the HMD.

17. The HMD of claim 16 , wherein responsive to making the determination that the trigger event occurred, (i) generating the first image, and (ii) causing the camera to capture the second image comprises responsive to making the determination that the trigger event occurred, substantially simultaneously (i) generating the first image, and (ii) causing the camera to capture the second image.

18. The HMD of claim 16 , wherein the generated first image comprises a screenshot of content displayed on the display.

19. The HMD of claim 16 , the set of functions further comprising:
responsive to making the determination that the trigger event occurred, determining, by the HMD, data relating to the HMD's operation proximate a time that the trigger event occurred, wherein the generated composite image further includes an indication of the determined data.

20. A non-transitory computer-readable medium having stored thereon, program instructions that when executed by a processor, cause performance of a set of acts comprising:
displaying content on a transparent or semi-transparent display of a head-mountable device (HMD), wherein the displayed content is perceived by a wearer of the HMD as being superimposed over a real-world field-of-view seen through the transparent or semi-transparent display;
making, by the HMD, a determination that a trigger event occurred;
responsive to making the determination that the trigger event occurred, both: (i) generating, by the HMD, a first image of the displayed content, and (ii) causing, by the HMD, a camera to capture a second image of the real-world field-of-view associated with the HMD; and
generating, by the HMD, a composite image that combines the generated first image, the captured second image, and a glass-texture effect image by overlaying the generated first image on the glass-texture effect image and overlaying the glass-texture effect image on the captured second image, wherein the glass-texture effect image is neither displayed on the display of the HMD nor part of the real-world field-of-view associated with the HMD.

Assignees

• Google Inc

Inventors

• The Richard
• Braun Max Benjamin
• Mckenzie Chris
• Chen Alexander Hanbing

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B27/017

  Head mounted · CPC title

• H04N7/188

  Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253) · CPC title