Augmentation of a gaming controller via projection system of an autonomous personal companion

Patent metadata

Field	Value
Publication number	US-10456682-B2
Application number	US-201715714591-A
Country	US
Kind code	B2
Filing date	Sep 25, 2017
Priority date	Sep 25, 2017
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

An autonomous personal companion executing a method of projection including tracking within a reference system a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the personal companion is configured to provide services to the user. The method includes receiving information related to the gaming application from a gaming console supporting the game play. The method includes generating content based on the information related to the gaming application. The method includes moving the autonomous personal companion to a location having a direct line of sight to the controller. The method includes projecting the content from the autonomous personal companion to a surface of the gaming controller.

First claim

What is claimed is:

1. A method of projection, comprising:
tracking within a reference system by an autonomous personal companion a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the autonomous personal companion is configured to provide services to the user;
receiving information related to the gaming application from a gaming console supporting the game play;
generating content based on the information related to the gaming application;
moving the autonomous personal companion to a location having a direct line of sight to the current position and orientation of the gaming controller; and
projecting the content from the autonomous personal companion to a surface of the gaming controller.

2. The method of claim 1 , further comprising:
tracking an orientation of the gaming controller using a vision system of the autonomous personal companion, wherein the vision system is configured to track the current position of the gaming controller;
determining a position and orientation of a displayable surface of the gaming controller; and
aligning the content to the orientation of the displayable surface of the gaming controller when projecting the content to the surface.

3. The method of claim 1 , wherein tracking a current position comprises:
augmenting the tracking of the current position of the gaming controller by a second tracking system of the gaming console.

4. The method of claim 1 , further comprising:
determining a current context in the gaming application associated with the game play;
based on the information, determining an actuator in the gaming controller that when actuated provides a command to the gaming application at a point; and
displaying the content adjacent to the actuator, wherein the content is a visual representation of the command.

5. The method of claim 4 , wherein the content comprises text or an icon.

6. The method of claim 4 , wherein the actuator comprises an analog stick or trigger button.

7. The method of claim 1 , further comprising:
generating the content comprising a visual actuator that when interacted with provides a command to the gaming application; and
projecting the visual actuator as the content onto the surface;
determining when the user actuates the visual actuator; and
sending the command to the gaming application for execution in the game play.

8. The method of claim 1 , wherein the generating content further comprises:
determining a current context in the game play based on the information; and
generating contextually relevant information as the content based on the current context.

9. A computer system comprising:
a processor; and
memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method comprising:
tracking within a reference system by an autonomous personal companion a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the autonomous personal companion is configured to provide services to the user;
receiving information related to the gaming application from a gaming console supporting the game play;
generating content based on the information related to the gaming application;
moving the autonomous personal companion to a location having a direct line of sight to the current position and orientation of the gaming controller; and
projecting the content from the autonomous personal companion to a surface of the gaming controller.

10. The computer system of claim 9 , wherein the method further comprises:
tracking an orientation of the gaming controller using a vision system of the autonomous personal companion, wherein the vision system is configured to track the current position of the gaming controller;
determining a position and orientation of a displayable surface of the gaming controller; and
aligning the content to the orientation of the displayable surface of the gaming controller when projecting the content to the surface.

11. The computer system of claim 9 , wherein the method further comprises:
determining a current context in the gaming application associated with the game play;
based on the information, determining an actuator in the gaming controller that when actuated provides a command to the gaming application at a point; and
displaying the content adjacent to the actuator, wherein the content is a visual representation of the command.

12. The computer system of claim 11 , wherein in the method the content comprises text or an icon.

13. The computer system of claim 9 , wherein the method further comprises:
generating the content comprising a visual actuator that when interacted with provides a command to the gaming application; and
projecting the visual actuator as the content onto the surface;
determining when the user actuates the visual actuator; and
sending the command to the gaming application for execution in the game play.

14. The computer system of claim 9 , wherein in the method the generating content further comprises:
determining a current context in the game play based on the information; and
generating contextually relevant information as the content based on the current context.

15. A non-transitory computer-readable medium storing a computer program for implementing a method, the computer-readable medium comprising:
program instructions for tracking within a reference system by an autonomous personal companion a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the autonomous personal companion is configured to provide services to the user;
program instructions for receiving information related to the gaming application from a gaming console supporting the game play;
program instructions for generating content based on the information related to the gaming application;
program instructions for moving the autonomous personal companion to a location having a direct line of sight to the current position and orientation of the gaming controller; and
program instructions for projecting the content from the autonomous personal companion to a surface of the gaming controller.

16. The non-transitory computer-readable medium of claim 15 , further comprising:
program instructions for tracking an orientation of the gaming controller using a vision system of the autonomous personal companion, wherein the vision system is configured to track the current position of the gaming controller;
program instructions for determining a position and orientation of a displayable surface of the gaming controller; and
program instructions for aligning the content to the orientation of the displayable surface of the gaming controller when projecting the content to the surface.

17. The non-transitory computer-readable medium of claim 15 , further comprising:
program instructions for determining a current context in the gaming application associated with the game play;
based on the information, program instructions for determining an actuator in the gaming controller that when actuated provides a command to the gaming application at a point; and
program instructions for displaying the content adjacent to the actuator, wherein the content is a visual representation of the command.

18. The non-transitory computer-readable medium of claim 17 , wherein the content comprises text or a

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Stafford Jeffrey Roger
• Fernandez-Rico Javier

Classifications

• A63F13/335

  using Internet · CPC title

• A63F13/32

  using local area network [LAN] connections · CPC title

• A63F13/24

  Constructional details thereof, e.g. game controllers with detachable joystick handles · CPC title

• A63F13/211

  using inertial sensors, e.g. accelerometers or gyroscopes · CPC title

• A63F13/52 Primary

  involving aspects of the displayed game scene · CPC title