Wearable electronic glasses display instructions as virtual hand gestures

Patent metadata

Field	Value
Publication number	US-11645816-B2
Application number	US-201916679258-A
Country	US
Kind code	B2
Filing date	Nov 10, 2019
Priority date	Oct 2, 2013
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

Wearable electronic glasses (WEG) of a first user display an image of an object on which a second user desires instructions how to complete a task. The WEG displays an augmented reality (AR) image of the object and captures hand gestures of the first user the completing the task on the AR image of the object. A smartphone of the second user displays the AR image and AR hands replicating movements of the hand gestures of the first user completing the task.

First claim

What is claimed is:

1. A method comprising:
displaying, on a display of wearable electronic glasses (WEG) worn on a head of a first user, an image of an object captured with a portable electronic device (PED) with a second user and an augmented reality (AR) image of the object;
capturing, with a camera in the WEG worn on the head of the first user, hand gestures of the first user interacting with the AR image of the object being displayed on the display of the WEG to show how to complete a task on the object; and
displaying, on a display of the PED with the second user, the object, the AR image of the object, and AR hands that replicate movements of the hand gestures of the first user interacting with the AR image of the object to show how to complete the task on the object.

2. The method of claim 1 further comprising:
moving where the AR hands appear on the display of the PED to another location on the display of the PED in response to a distance between the AR hands and a perimeter of the display of the PED decreasing.

3. The method of claim 1 further comprising:
preventing the AR hands from moving off the display of the PED when the second user moves the PED while capturing an image of the object by moving the AR hands to another location on the display of the PED where more space exists.

4. The method of claim 1 further comprising:
switching locations where the AR hands are displayed on the display of the PED when a distance between the AR hands and the object being displayed on the display of the PED decreases.

5. The method of claim 1 further comprising:
preventing a perimeter of a field of view of the PED from colliding with the AR hands by reducing a size of the AR hands being displayed on the display of the PED, wherein the PED is a smartphone.

6. The method of claim 1 further comprising:
tracking a direction of gaze of the second user; and
automatically displaying the AR hands in response to determining that the direction of gaze of the second user is directed to the object.

7. The method of claim 1 further comprising:
tracking a direction of gaze of the second user; and
automatically displaying written instructions describing how to complete the task on the object in response to determining that the direction of gaze of the second user is directed to the object.

8. The method of claim 1 further comprising:
tracking a location of the second user with respect to the object; and
automatically displaying the AR hands in response to determining that the location of the second user is within a predetermined number of feet of the object.

9. A non-transitory computer readable storage medium storing instructions that cause one or more electronic devices to execute a method comprising:
displaying, on a display of wearable electronic glasses (WEG) worn on a head of a first user, an object captured with a camera of a wearable electronic device (WED) of a second user and an augmented reality (AR) image of the object;
capturing, with a camera in the WEG, hand gestures of the first user interacting with the AR image of the object being displayed on the display of the WEG to show how to complete a task on the object; and
simultaneously displaying, on a display of the WED of the second user, the AR image of the object and AR hands that replicate movements of the hand gestures of the first user that show how to complete the task on the AR image of the object being displayed on the display of the WED.

10. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
displaying, on the display of the WEG, virtual hands of the first user that move to complete the task on the AR image of the object; and
moving the virtual hands to another location on the display of the WEG when an available space in a field of view of the WEG becomes too small to include the virtual hands.

11. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
displaying, on the display of the WEG, the object and virtual hands of the first user; and
switching locations of the virtual hands with respect to the object within a field of view of the WEG to compensate for movement of the WEG.

12. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
displaying, on the display of the WEG, virtual hands of the first user; and
increasing a length of time that the virtual hands remain present in a field of view of WEG by moving the virtual hands to a different location within the field of view in response to an edge of the field of view moving to collide with the virtual hands.

13. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
preventing a perimeter of a field of view of the WEG from colliding with the AR image of the object being displayed on the display of the WEG by moving the AR image of the object to another location on the display of the WEG where more available space exists to display the AR image of the object.

14. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
tracking, with the WEG, a direction of gaze of the first user; and
automatically displaying virtual hands of the first user when the direction of gaze of the first user is directed to the AR image of the object being displayed on the display of the WEG.

15. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
tracking, with the WEG, a direction of focus of the first user; and
automatically displaying the AR image of the object when the direction of focus of the first user is directed on the object.

16. Wearable electronic glasses (WEG) worn on a head of a first user, comprising:
a wireless interface that receives an image of an object captured with a portable electronic device (PED) with a second user requesting instructions to complete a task on the object;
a display that displays an augmented reality (AR) image of the object and AR hands that show hand gestures interacting with the AR image of the object to show the first user completing a task on the AR image of the object; and
a camera that captures the hand gestures of the first user interacting with the AR image of the object being displayed on the display of the WEG to capture the first user completing the task on the AR image of the object,
wherein the wireless interface transmits, to the PED with the second user, the hand gestures of the first user interacting with the AR image of the object to show the second user how to complete the task on the object.

17. The WEG of claim 16 further comprising:
eye tracking that tracks a direction of gaze of the first user, wherein the display displays the AR image of the object and the AR hands when the direction of gaze of the first user is directed to the image of the object displayed on the display.

18. The WEG of claim 16 further comprising:
eye tracking that tracks a direction of gaze of the first user, wherein the display displays an AR message from the second user when the direction of gaze of the first user looks in a direction that includes the object.

19. The WEG of claim 16 further comprising:
a global positioning system (GPS) sensor that determines a location of the WEG, wherein the display displays an AR message from the second u

Assignees

• Lyren Philip Scott

Inventors

• Lyren Philip Scott

Classifications

• G02B27/0172

  characterised by optical features · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B2027/0141

  characterised by the informative content of the display · CPC title

• G06F40/51

  Translation evaluation · CPC title