Method and system for pervasive memory augmentation via a heads-up display

Patent metadata

Field	Value
Publication number	US-11513355-B2
Application number	US-202016810323-A
Country	US
Kind code	B2
Filing date	Mar 5, 2020
Priority date	Mar 13, 2019
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

A method of pervasive memory augmentation includes receiving a request to augment a memory via a heads-up display having a field of view and generating one or more processor-readable memory cues that are connected to the memory. A cued content is generated based on the one or more processor-readable memory cues. The cued content is presented in a display space in the field of view of the heads-up display on a spaced repetition schedule.

First claim

The invention claimed is:

1. A method of pervasive memory augmentation, the method comprising:
in response to a request to augment a memory received via a wearable heads-up display having a field of view, generating one or more processor-readable memory cues that are connected to the memory;
generating a cued content based on the one or more processor-readable memory cues;
presenting the cued content in a display space in the field of view of the wearable heads-up display on a spaced repetition schedule; and
pausing presentation of the cued content when notifications are not allowed including at least one of: when a user of the wearable heads-up display is engaged in a face-to-face conversation with another person; at designated times of day; and when the user has set a “do not disturb” mode.

2. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule comprises presenting the cued content at one of (i) equally spaced time intervals over a set time period, (ii) unequally spaced time intervals over a set time period, and (iii) progressively expanding time intervals over a set time period.

3. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule begins automatically following generation of the one or more processor-readable memory cues.

4. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule begins in response to a subsequent request to begin augmenting the memory.

5. The method of claim 1 , further comprising apart from presenting the cued content in the display space on the spaced repetition schedule, presenting the cued content in the display space in response to at least one of (a) a request to present the memory cues, (b) a reminder of an event linked to the memory, and (c) detecting that the wearable heads-up display is at a location linked to the memory.

6. The method of claim 1 , further comprising repeating presenting the cued content in the display space on the spaced repetition schedule in response to a request to replay augmentation of the memory.

7. The method of claim 1 , further comprising acquiring memory data that describes the memory prior to generating the one or more processor-readable memory cues that are connected to the memory.

8. The method of claim 7 , wherein acquiring memory data comprises receiving audio data captured in a portion of an environment of the wearable heads-up display by at least one microphone during a period in which the memory is made.

9. The method of claim 8 , wherein generating one or more processor-readable memory cues that are connected to the memory comprises:
causing conversion of the audio data to text data;
causing natural language processing of the text data to identify one or more topics and/or keywords in the text data; and
causing generation of the one or more processor-readable memory cues from the one or more topics and/or keywords.

10. The method of claim 9 , further comprising receiving the audio data from the at least one microphone by a processor of the wearable heads-up display and transmitting the audio data to a remote device, wherein a remote processor associated with the remote device generates the one or more processor-readable memory cues.

11. The method of claim 1 , further comprising receiving a location information of one or more digital assets connected to the memory and retrieving the one or more digital assets using the location information, wherein generating one or more processor-readable memory cues that are connected to the memory comprises generating the one or more processor-readable memory cues from the one or more digital assets.

12. The method of claim 11 , wherein receiving a location information of one or more digital assets connected to the memory comprises presenting a user interface in the display space to receive the location information.

13. The method of claim 11 , wherein retrieving the one or more digital assets using the location information comprises at least one of retrieving a digital asset from an application installed on a host device paired with the wearable heads-up display, retrieving a digital asset from an application hosted on at least one remote server, and retrieving a digital asset from a posting on a social network.

14. The method of claim 1 , wherein presenting the cued content in the display space on the spaced repetition schedule comprises modulating an output light of a display light source with at least a portion of the cued content and projecting the output light to the display space.

15. The method of claim 1 , wherein the memory indicated in the request to augment a memory is selected from a memory of a meeting, a memory of a past event, a memory of a future event, a memory of learning material, and a memory of sensitive information.

16. A method of pervasive memory augmentation, the method comprising:
in response to a request to augment a memory received via a wearable heads-up display having a field of view, generating one or more processor-readable memory cues that are connected to the memory;
generating a cued content based on the one or more processor-readable memory cues;
presenting the cued content in a display space in a field of view of the wearable heads-up display in response to at least one of (a) a request to present the memory cues, (b) a reminder of an event linked to the memory, and (c) detecting that the wearable heads-up display is at a location linked to the memory; and
pausing presentation of the cued content when notifications are not allowed including at least one of: when a user of the wearable heads-up display is engaged in a face-to-face conversation with another person; at designated times of day; and when the user has set a “do not disturb” mode.

17. A system for pervasive memory augmentation, the system comprising:
a wearable heads-up display having a field of view, the wearable heads-up display comprising a display light source, a processor communicatively coupled to the display light source, and a non-transitory processor-readable medium communicatively coupled to the processor, wherein the non-transitory processor-readable medium stores data and/or processor-executable instructions that, when executed by the processor, causes the wearable heads-up display to:
receive a request to augment a memory;
generate one or more processor-readable memory cues that are connected to the memory;
generate a cued content based on the one or more processor-readable memory cues;
present the cued content in a display space in the field of view of the wearable heads-up display on a spaced repetition schedule; and
pause presentation of the cued content when notifications are not allowed including at least one of: when a user of the wearable heads-up display is engaged in a face-to-face conversation with another person; at designated times of day; and when the user has set a “do not disturb” mode.

18. The system of claim 17 , further comprising at least one sensor to capture a sensory input in an environment of the wearable heads-up display.

19. The system of claim 17 , further comprising a support frame that in use is worn on a head of a user, wherein the display light source and processor are carried by the support frame.

20. The system of claim 17 , wherein the display light source is a scanning laser projector, a micro-display, or a digital light processing projector.

Assignees

• North Inc
• Google Llc

Inventors

• Niforatos Evangelos
• Perry Daniel

Classifications

• G02B2027/0174

  holographic · CPC title

• G06F40/106

  Display of layout of documents; Previewing · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G06F40/10

  Text processing (natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40) · CPC title