Wearable electronic glasses display an augmented reality (AR) image to show a task

Patent metadata

Field	Value
Publication number	US-11657577-B2
Application number	US-201916679263-A
Country	US
Kind code	B2
Filing date	Nov 10, 2019
Priority date	Oct 2, 2013
Publication date	May 23, 2023
Grant date	May 23, 2023

Abstract

A wearable electronic device (WED) worn on a head of a first user displays a virtual location with virtual objects. The WED receives a task on one of the AR objects that corresponds to a real object at a real location. Wearable electronic glasses (WEG) worn on a head of a second user display an augmented reality (AR) image that shows the task to be performed on the real object.

First claim

What is claimed is:

1. A method comprising:
displaying, on a display of a portable electronic device (PED) in a hand of a first user, a virtual location with virtual objects that emulates a real location with real objects that is remotely located from the PED;
tracking, with the PED in the hand of the first user, hand movements of the first user interacting with one of the virtual objects being displayed on the display of the PED such that the hand movements interacting with the one of the virtual objects show a task to be performed on one of the real objects that corresponds to the one of the virtual objects;
tracking, with wearable electronic glasses (WEG) worn on a head of a second user located at the real location, a direction of gaze of the second user; and
displaying, with the WEG worn on the head of the second user and in response to the direction of gaze of the second user being at the one of the real objects, an augmented reality (AR) image of the hand movements and the virtual objects that shows the task to be performed that was provided by the first user on the one of the virtual objects.

2. The method of claim 1 further comprising:
receiving, at a user interface (UI) of the PED and from the first user, an action that highlights the one of the virtual objects; and
displaying, with the WEG worn on the head of the second user and in response to the action from the first user, the AR image that highlights the one of the real objects when the direction of gaze of the second user is at the one of the real objects.

3. The method of claim 1 further comprising:
receiving, at the one of the virtual objects being displayed on the display of the PED and from the first user, an action that moves the one of the virtual objects from a first location at the virtual location to a second location at the virtual location; and
displaying, with the WEG worn on the head of the second user and in response to the action from the first user, the AR image that moves from a first location at the real location to a second location at the real location,
wherein the first location at the real location corresponds to the first location at the virtual location, and the second location at the real location correspond to the second location at the virtual location.

4. The method of claim 1 further comprising:
receiving, at the one of the virtual objects being displayed on the display of the PED and from the first user, the task that includes a sequence of actions showing where to move the one of the real objects at the real location; and
displaying, with the WEG worn on the head of the second user, the AR image that moves to different locations at the real location and shows the sequence of actions showing where to move the one of the real objects.

5. The method of claim 1 further comprising:
displaying, on the display of the PED in the hand of the first user, video of the real location received from a camera in the WEG;
receiving, at a user interface (UI) of the PED and from the first user, an action that selects a real object shown in the video of the real location; and
displaying, with the WEG, the AR image over the real object to visually identify the real object selected by the first user.

6. The method of claim 1 further comprising:
receiving, at a user interface (UI) of the PED, an action from the first user that moves the one of the virtual objects; and
moving, with the WEG, the AR image in real-time with movements of the one of the virtual objects to show the task to be performed.

7. The method of claim 1 further comprising:
moving a location where the AR image is displayed with the WEG in order to avoid a collision between the AR image and a field of view of the WEG.

8. The method of claim 1 further comprising:
moving a location of the AR image from one side of the one of the real objects to an opposite side of the one of the real objects in response to movement of the WEG that causes an available space on the one side of the real objects to be too small for the AR image.

9. A non-transitory computer readable storage medium storing instructions that cause one or more electronic devices to execute a method comprising:
displaying, with a wearable electronic device (WED) worn on a head of a first user, a virtual location having virtual objects that emulates a real location having real objects;
receiving, at one of the virtual objects being displayed with the WED and from the first user, hand gestures of the first user interacting with the one of the virtual objects such that the hand gestures interacting with the one of the virtual objects show a task to be performed on one of the real objects that corresponds to the one of the virtual objects; and
displaying, with wearable electronic glasses (WEG) worn on a head of a second user, an augmented reality (AR) image of the hand gestures that moves at the real location and shows the task to be performed on the one of the real objects that was provided by the first user on the one of the virtual objects.

10. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
tracking, with the WEG worn on the head of the second user located at the real location, a direction of gaze of the second user; and
displaying, with the WEG worn on the head of the second user located at the real location, the AR image over the one of the real objects in response to the direction of gaze of the second user being at the one of the real objects.

11. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
moving, with the WEG worn on the head of the second user, the AR image to correspond with real-time movements of the one of the virtual objects being displayed on the WED and being moved by the first user.

12. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
highlighting, with the WEG worn on the head of the second user, the one of real objects with the AR image in response to a user interface (UI) of the WED receiving highlighting of the one of the virtual objects being displayed with the WED.

13. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
receiving, at the one of the virtual objects being displayed on the WED and from the first user, the task that includes a sequence of actions showing where to move the one of the real objects at the real location; and
displaying, with the WEG worn on the head of the second user, the AR image that moves to different locations at the real location and shows the sequence of actions showing where to move the one of the real objects.

14. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
displaying, with the WED, the virtual location having the virtual objects in virtual reality (VR);
capturing, with a camera in the WEG worn on the head of the second user, video of the real location; and
displaying, with the WED worn on the head of the first user, the video of the real location.

15. The non-transitory computer readable storage medium storing instructions of claim 9 in which the method further comprises:
moving a location where the AR image is displayed with the WEG in order to avoid a collision between the AR image and a field of view of the WEG.

16. An electronic system, comprising:
a wearable electronic device (WED) worn on a head of a first user a

Assignees

• Lyren Philip Scott

Inventors

• Lyren Philip Scott

Classifications

• G06F3/012

  Head tracking input arrangements · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G02B27/0172

  characterised by optical features · CPC title