Exercising application for personal audio/visual system

Patent metadata

Field	Value
Publication number	US-9355583-B2
Application number	US-201414472267-A
Country	US
Kind code	B2
Filing date	Aug 28, 2014
Priority date	Sep 30, 2011
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

The technology described herein includes a see-through, near-eye, mixed reality display device for providing customized experiences for a user. The personal A/V apparatus serves as an exercise program that is always with the user, provides motivation for the user, visually tells the user how to exercise, and lets the user exercise with other people who are not present.

First claim

What is claimed is:

1. A method for presenting a personalized experience using a personal see-through A/V apparatus, comprising:
accessing a first exercise routine for a first person;
accessing data for a second person for a second exercise routine different from the first exercise routine, wherein the second exercise routine is associated with a location that is remote from the personal see-through A/V apparatus;
estimating a performance of how the second person would perform the first exercise routine based on the data; and
presenting a virtual image of someone performing the first exercise routine based on the estimated performance so that the first person can see the virtual image inserted into a real scene viewed through the personal see-through A/V apparatus as the first person performs the first exercise routine.

2. The method of claim 1 , wherein:
the estimated performance is based on past performance of the second exercise routine.

3. The method of claim 1 , wherein:
the estimated performance is based on a live performance of the second exercise routine.

4. The method of claim 1 , wherein the presenting a virtual image of someone performing the first exercise routine based on the estimated performance includes:
presenting an avatar of the second person that integrates the second person into an environment of the first person.

5. The method of claim 4 , wherein the second person is exercising at the remote location from the personal see-through A/V apparatus.

6. The method of claim 1 , wherein the accessing data for the second person for the second exercise routine different from the first exercise routine includes:
accessing real time exercise data for the second person at the location that is remote from the personal see-through A/V apparatus.

7. The method of claim 6 , wherein the presenting a virtual image of someone performing the first exercise routine based on the estimated performance includes:
presenting the virtual image based on the real time exercise data so that the first person can see a representation of the second person appear to exercise in the same 3D coordinate system with the first person.

8. The method of claim 1 , wherein the presenting the virtual image of someone performing the first exercise routine based on the estimated performance includes:
rendering an avatar representing the second person's performance synchronized in time and space with the first person, the rendering being performed on the personal see-through A/V apparatus so that the first person can see the avatar inserted into a real scene viewed through the personal see-through A/V apparatus.

9. A system, comprising:
a see-through, near-eye, augmented reality display;
a sensor;
processing logic in communication with the sensor and the augmented reality display, wherein the processing logic is configured to:
access a first exercise routine associated with a first location of the see-through, near-eye, augmented reality display, wherein a first person is associated with the see-through, near-eye, augmented reality display;
access data for a second person for a second exercise routine different from the first exercise routine, the second exercise routine associated with a second location that is remote from the first location;
estimate a performance of how the second person would perform the first exercise routine based on the data; and
present a virtual image of someone performing the first exercise routine based on the estimated performance so that the first person can see the virtual image inserted into a real scene viewed through the augmented reality display when the see-through, near-eye, augmented reality display is at the first location associated with the first exercise routine.

10. The system of claim 9 , wherein the processing logic is configured to receive as input the first exercise routine.

11. The system of claim 9 , wherein the first exercise routine is a first route and the second exercise routine is a second route.

12. The system of claim 9 , wherein the processing logic is configured to estimate the performance based on a live performance of the second exercise routine by the second person.

13. The system of claim 9 , wherein the processing logic is configured to:
access real time exercise data for the second person at the second location that is remote from first location; and
estimate the performance of how the second person would perform the first exercise routine based on the real time exercise data.

14. The system of claim 9 , wherein the processing logic comprises a server in communication with the see-through, near-eye, augmented reality display.

15. A personal see-through A/V apparatus, comprising:
a see-through, near-eye, augmented reality display;
a sensor; and
a processor in communication with the sensor and the augmented reality display, wherein the processor is configured to:
access a first course of action for a first person, wherein the first course of action is for a present location of the personal see-through A/V apparatus;
access data for a second person for a second course of action different from the first course of action, the second course of action is for a remote physical location from the personal see-through A/V apparatus;
estimate a performance of how the second person would perform the first course of action based on the accessed data; and
present a virtual image of someone performing the first course of action based on the estimated performance so that the first person can see the virtual image inserted into a real scene viewed through the personal see-through A/V apparatus when the personal see-through A/V apparatus is at the present location.

16. The personal see-through A/V apparatus of claim 15 , wherein the processor is configured to estimate the performance based on a live performance of the second course of action.

17. The personal see-through A/V apparatus of claim 16 , wherein the processor is configured to present the virtual image based on the live performance so that the first person can see a representation of the second person appear to exercise in the same 3D coordinate system with the first person.

18. The personal see-through A/V apparatus of claim 15 , wherein the processor is configured to render an image representing the second person's performance synchronized in time and space with the first person, wherein the processor is configured to render on the personal see-through A/V apparatus so that the first person can see the virtual image inserted into a real scene viewed through the personal see through A/V apparatus.

19. The personal see-through A/V apparatus of claim 15 , wherein the virtual image is based on real time video data.

20. The personal see-through A/V apparatus of claim 15 , wherein to estimate the performance the processor is configured to:
access 3D positional data for the second person; and
determine speed and direction from the 3D positional data.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Geisner Kevin A
• Perez Kathryn Stone
• Latta Stephen G
• Sugden Ben J
• Vaught Benjamin I
• Kipman Alex Aben-Athar

Classifications

• A63F2300/8005

  Athletics · CPC title

• A63F2300/695

  Imported photos, e.g. of the player · CPC title

• A63F13/816

  Athletics, e.g. track-and-field sports · CPC title

• A63F13/655

  by importing photos, e.g. of the player · CPC title

• A63F13/26

  having at least one additional display device, e.g. on the game controller or outside a game booth · CPC title