Position-dependent gaming, 3-D controller, and handheld as a remote

Patent metadata

Field	Value
Publication number	US-9159165-B2
Application number	US-83567110-A
Country	US
Kind code	B2
Filing date	Jul 13, 2010
Priority date	Jul 13, 2010
Publication date	Oct 13, 2015
Grant date	Oct 13, 2015

Abstract

Methods and systems for using a position of a mobile device with an integrated display as an input to a video game or other presentation are presented. Embodiments include rendering an avatar on a mobile device such that it appears to overlay a competing user in the real world. Using the mobile device's position, view direction, and the other user's mobile device position, an avatar (or vehicle, etc.) is depicted at an apparently inertially stabilized location of the other user's mobile device or body. Some embodiments may estimate the other user's head and body positions and angles and reflect them in the avatar's gestures.

First claim

What is claimed is:

1. A method for augmenting video, comprising:
receiving a first position coordinate corresponding to a first user, the first position coordinate relative to a first video display;
receiving a first view direction corresponding to the first user, the first view direction relative to the first video display;
receiving a second position coordinate corresponding to a second user, the second position coordinate relative to a second video display;
estimating a position of the second user's head relative to a first mobile device;
determining a direction and range from the first position coordinate to the second position coordinate; and
rendering, on an integrated display of the first mobile device, an object based on the determined direction and range from the first position coordinate to the second position coordinate and based on the received first view direction and based on the estimated position of the second user's head, wherein the first video display and the second video display are different from each other and from the integrated display of the first mobile device.

2. The method of claim 1 wherein the object is displayed as if it were co-located with the second user from a vantage point of the first user.

3. The method of claim 1 further comprising:
receiving a second view direction corresponding to the second user, the second view direction relative to the second video display,
wherein the rendering is further based on the received second view direction.

4. The method of claim 3 wherein the object is selected from the group consisting of an avatar and a vehicle.

5. The method of claim 1 wherein the position coordinates corresponding to the users are estimates of where the users' heads or hands are.

6. The method of claim 5 wherein the first position coordinate corresponding to the first user includes an estimate of position of the first user's hands relative to the first mobile device.

7. The method of claim 1 wherein the position coordinates corresponding to the users are position coordinates of mobile devices worn by the users.

8. The method of claim 4 wherein the first view direction comprises an azimuth and elevation.

9. The method of claim 1 wherein slewing the first mobile device to a first direction causes the rendered object to disappear off to an opposite direction of the first direction.

10. The method of claim 1 wherein the first view direction comprises a bearing direction in relation to a fixed point.

11. The method of claim 1 further comprising:
rendering, on the integrated display of the first mobile device, a background object.

12. The method of claim 1 wherein the position of the second user's head relative to a first mobile device is estimated using predefined models.

13. A system for augmented video, the system comprising:
a processor; and
a memory device including instructions that, when executed by the processor, cause the processor to:
receive a first position coordinate corresponding to a first user, the first position coordinate relative to a first video display;
receive a first view direction corresponding to the first user, the first view direction relative to the first video display;
receive a second position coordinate corresponding to a second user, the second position coordinate relative to a second video display;
estimate a position of the second user's head relative to a first mobile device;
determine a direction and range from the first position coordinate to the second position coordinate; and
render, on an integrated display of the first mobile device, an object based on the determined direction and range from the first position coordinate to the second position coordinate and based on the received first view direction and based on the estimated position of the second user's head, wherein the first video display and the second video display are different from each other and from the integrated display of the first mobile device.

14. The system of claim 13 wherein the object is displayed as if it were co-located with the second user from a vantage point of the first user.

15. The system of claim 13 wherein the memory further includes instructions that, when executed by the processor, cause the processor to:
receive a second view direction corresponding to the second user, the second view direction relative to the second video display,
wherein the rendering is further based on the received second view direction.

16. The system of claim 13 wherein the memory further includes instructions that, when executed by the processor, cause the processor to:
send a remote control command to the first video display.

17. A machine-readable non-transitory storage medium embodying information indicative of instructions for causing one or more machines to perform operations, the operations comprising:
receiving a first position coordinate corresponding to a first user, the first position coordinate relative to a first video display;
receiving a first view direction corresponding to the first user, the first view direction relative to the first video display;
receiving a second position coordinate corresponding to a second user, the second position coordinate relative to a second video display;
estimating a position of the second user's head relative to a first mobile device;
determining a direction and range from the first position coordinate to the second position coordinate; and
rendering, on an integrated display of the first mobile device, an object based on the determined direction and range from the first position coordinate to the second position coordinate and based on the received first view direction and based on the estimated position of the second user's head, wherein the first video display and the second video display are different from each other and from the integrated display of the first mobile device.

18. The machine-readable non-transitory storage medium of claim 17 wherein the position coordinates corresponding to the users are position coordinates of mobile devices worn by the users.

19. The machine-readable non-transitory storage medium of claim 17 wherein the first view direction comprises an azimuth and elevation.

20. The machine-readable non-transitory storage medium of claim 17 wherein the first mobile device tracks the first video display using markers on the first video display.

Assignees

• Stafford Jeffrey R
• Zhu Yunpeng
• Osman Steven
• Sony Computer Entertainment Inc

Inventors

• Stafford Jeffrey R
• Zhu Yunpeng
• Osman Steven

Classifications

• A63F2300/204

  the platform being a handheld device · CPC title

• A63F2300/105

  using inertial sensors, e.g. accelerometers, gyroscopes · CPC title

• A63F2300/5553

  user representation in the game field, e.g. avatar · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title