Information processor, information processing method and program

Patent metadata

Field	Value
Publication number	US-9058057-B2
Application number	US-201113049445-A
Country	US
Kind code	B2
Filing date	Mar 16, 2011
Priority date	Mar 29, 2010
Publication date	Jun 16, 2015
Grant date	Jun 16, 2015

Abstract

Disclosed herein is an information processor including, a storage section configured to store data representing the appearance features of a target device and data of an operation image used for operation of the target device, an acquisition section configured to acquire an image captured by a camera, a recognition section configured to recognize an object included in the image based on the feature quantity data stored in the storage section, and a display control section configured to display the operation image based on the data of the operation image stored in the storage section if the object included in the image is recognized by the recognition section as the target device.

First claim

What is claimed is:

1. An information processor comprising:
a storage unit configured to store data representing the appearance features of a separate target device and data of a graphical user interface image capable of being manipulated by a user to operate the separate target device;
an acquisition unit configured to acquire an image captured by a camera;
a recognition unit configured to recognize the separate target device included in the image based on the appearance features stored in the storage unit;
a display control unit configured to display the graphical user interface image based on the data of the graphical user interface image stored in the storage unit if the separate target device included in the image is recognized by the recognition unit;
a finger recognition unit configured to recognize a finger of the user included in the image captured by the camera; and
a control unit configured to transmit a remote control command to the separate target device if the position of the finger at least partially interacts with the graphical user interface image to instruct an operation to be performed by the separate target device associated with the graphical user interface image.

2. The information processor of claim 1 , wherein
the display control unit displays the graphical user interface image at the position of a display that is at least overlapping with the target device.

3. The information processor of claim 1 , wherein
the finger recognition unit further recognizes the size of the finger of the user included in the image, and
the control unit transmits the command when the finger of the user in the image is located at the position corresponding to the position of the graphical user interface image and when, at that position, the in-image size of the finger of the user becomes larger again after having become smaller than immediately before.

4. The information processor of claim 1 , wherein
the finger recognition unit further recognizes the in-image size of the finger of the user included in the image, and wherein
the control unit transmits the command when the finger of the user in the image is located at the position corresponding to the position of the graphical user interface image and when, at that position, the in-image size of the finger of the user becomes smaller than immediately before.

5. The information processor of claim 1 still further comprising:
a setting information storage unit configured to store setting information representing the details of settings related to the display of the graphical user interface image, wherein
the display control unit controls the display of the graphical user interface image according to the details of the setting information.

6. An information processing method comprising the steps of:
storing data representing the appearance features of a separate target device and data of a graphical user interface image capable of being manipulated by a user to operate the separate target device;
acquiring an image captured by a camera;
recognizing the separate target device included in the image based on the appearance features stored in the storage unit;
displaying the operation image based on the data of the graphical user interface image stored in the storage unit if the separate target device included in the image is recognized by the recognition unit;
recognizing a finger of the user included in the image captured by the camera; and
transmitting a remote control command to the separate target device if the position of the finger at least partially interacts with the graphical user interface image to instruct an operation to be performed by the separate target device associated with the graphical user interface image.

7. A program causing a computer to perform a process, the process comprising the steps of:
storing data representing the appearance features of a separate target device and data of a graphical user interface image capable of being manipulated by a user to operate the separate target device;
acquiring an image captured by a camera;
recognizing the separate target device included in the image based on the appearance features stored in the storage unit;
displaying the graphical user interface image based on the data of the operation image stored in the storage unit if the separate target device included in the image is recognized by the recognition unit;
recognizing a finger of the user included in the image captured by the camera; and
transmitting a remote control command to the separate target device if the position of the finger at least partially interacts with the graphical user interface image to instruct an operation to be performed by the separate target device associated with the graphical user interface image.

8. The information processor of claim 1 , wherein the graphical user interface image is displayed in three-dimensional augmented reality at a distance between the user and the separate target device.

9. The information processor of claim 1 , wherein a command input by the user to the graphical user interface image is transmitted from the information processor to the separate target device.

10. The information processor of claim 1 , wherein the separate target device is an electronically controlled device capable of receiving input from the information processor.

11. The information processor of claim 1 , wherein the separate target device is capable of being controlled by a separate controller capable of receiving input from the information processor.

Assignees

• Matsuda Kouichi
• Sony Corp

Inventors

• Matsuda Kouichi

Classifications

• G02B27/01

  Head-up displays · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G02B2027/0174

  holographic · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title