Augmentation of a gaming controller via projection system of an autonomous personal companion

Patent metadata

Field	Value
Publication number	US-2019091572-A1
Application number	US-201715714591-A
Country	US
Kind code	A1
Filing date	Sep 25, 2017
Priority date	Sep 25, 2017
Publication date	Mar 28, 2019
Grant date	—

Abstract

An autonomous personal companion executing a method of projection including tracking within a reference system a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the personal companion is configured to provide services to the user. The method includes receiving information related to the gaming application from a gaming console supporting the game play. The method includes generating content based on the information related to the gaming application. The method includes moving the autonomous personal companion to a location having a direct line of sight to the controller. The method includes projecting the content from the autonomous personal companion to a surface of the gaming controller.

First claim

What is claimed is:

1 . A method of projection, comprising:
tracking within a reference system by an autonomous personal companion a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the personal companion is configured to provide services to the user; receiving information related to the gaming application from a gaming console supporting the game play; generating content based on the information related to the gaming application; moving the autonomous personal companion to a location having a direct line of sight to the controller; and projecting the content from the autonomous personal companion to a surface of the gaming controller.

2 . The method of claim 1 , further comprising:
tracking an orientation of the gaming controller using a vision system of the personal companion, wherein the vision system is configured to track the current position of the gaming controller; determining a position and orientation of a displayable surface of the controller; and aligning the content to the orientation of the displayable surface of the controller when projecting the content to the surface.

3 . The method of claim 1 , wherein tracking a current position comprises:
augmenting the tracking of the current position of the gaming controller by a second tracking system of the gaming console.

4 . The method of claim 1 , further comprising:
determining a current context in the gaming application associated with the game play; based on the information, determining an actuator in the gaming controller that when actuated provides a command to the gaming application at the point; and displaying the content adjacent to the actuator, wherein the content is a visual representation of the command.

5 . The method of claim 4 , wherein the content comprises text or an icon.

6 . The method of claim 4 , wherein the actuator comprises an analog stick or trigger button.

7 . The method of claim 1 , further comprising:
generating the content comprising a visual actuator that when interacted with provides a command to the gaming application; and projecting the visual actuator as the content onto the surface; determining when the user actuates the visual actuator; and sending the command to the gaming application for execution in the game play.

8 . The method of claim 1 , wherein the generating content further comprises:
determining a current context in the game play based on the information; and generating contextually relevant information as the content based on the current context.

9 . A computer system comprising:
a processor; and memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method comprising:
tracking within a reference system by an autonomous personal companion a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the personal companion is configured to provide services to the user;
receiving information related to the gaming application from a gaming console supporting the game play;
generating content based on the information related to the gaming application;
moving the autonomous personal companion to a location having a direct line of sight to the controller; and
projecting the content from the autonomous personal companion to a surface of the gaming controller.

10 . The computer system of claim 9 , wherein the method further comprises:
tracking an orientation of the gaming controller using a vision system of the personal companion, wherein the vision system is configured to track the current position of the gaming controller; determining a position and orientation of a displayable surface of the controller; and aligning the content to the orientation of the displayable surface of the controller when projecting the content to the surface.

11 . The computer system of claim 9 , wherein the method further comprises:
determining a current context in the gaming application associated with the game play; based on the information, determining an actuator in the gaming controller that when actuated provides a command to the gaming application at the point; and displaying the content adjacent to the actuator, wherein the content is a visual representation of the command.

12 . The computer system of claim 11 , wherein in the method the content comprises text or an icon.

13 . The computer system of claim 9 , wherein the method further comprises:
generating the content comprising a visual actuator that when interacted with provides a command to the gaming application; and projecting the visual actuator as the content onto the surface; determining when the user actuates the visual actuator; and sending the command to the gaming application for execution in the game play.

14 . The computer system of claim 9 , wherein in the method the generating content further comprises:
determining a current context in the game play based on the information; and generating contextually relevant information as the content based on the current context.

15 . A non-transitory computer-readable medium storing a computer program for implementing a method, the computer-readable medium comprising:
program instructions for tracking within a reference system by an autonomous personal companion a current position and orientation of a gaming controller used for controlling game play of a gaming application played by a user, wherein the personal companion is configured to provide services to the user; program instructions for receiving information related to the gaming application from a gaming console supporting the game play; program instructions for generating content based on the information related to the gaming application; program instructions for moving the autonomous personal companion to a location having a direct line of sight to the controller; and program instructions for projecting the content from the autonomous personal companion to a surface of the gaming controller.

16 . The non-transitory computer-readable medium of claim 15 , further comprising:
program instructions for tracking an orientation of the gaming controller using a vision system of the personal companion, wherein the vision system is configured to track the current position of the gaming controller; program instructions for determining a position and orientation of a displayable surface of the controller; and program instructions for aligning the content to the orientation of the displayable surface of the controller when projecting the content to the surface.

17 . The non-transitory computer-readable medium of claim 15 , further comprising:
program instructions for determining a current context in the gaming application associated with the game play; based on the information, program instructions for determining an actuator in the gaming controller that when actuated provides a command to the gaming application at the point; and program instructions for displaying the content adjacent to the actuator, wherein the content is a visual representation of the command.

18 . The non-transitory computer-readable medium of claim 17 , wherein the content comprises text or an icon.

19 . The non-transitory computer-readable medium of claim 15 , further comprising:
program instructions for generating the content comprising a visual actuat

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Stafford Jeffrey Roger
• Fernandez-Rico Javier

Classifications

• A63F13/211

  using inertial sensors, e.g. accelerometers or gyroscopes · CPC title

• A63F13/24

  Constructional details thereof, e.g. game controllers with detachable joystick handles · CPC title

• G06T15/20 Primary

  Perspective computation · CPC title

• A63F13/52 Primary

  involving aspects of the displayed game scene · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title