Information processing apparatus and information processing method

Patent metadata

Field	Value
Publication number	US-10719121-B2
Application number	US-201414772046-A
Country	US
Kind code	B2
Filing date	Jan 20, 2014
Priority date	Mar 8, 2013
Publication date	Jul 21, 2020
Grant date	Jul 21, 2020

Abstract

An information processing apparatus includes circuitry configured to provide a user interface to a user, determine whether a predetermined object is present in an input recognition area of a sensor, and determine a region of interest within the user interface. The circuitry is further configured to determine, while the predetermined object is determined to be present in the input recognition area, whether the region of interest within the user interface changes. The circuitry is further configured to perform an operation based on whether the region of interest is determined, while the predetermined object is determined to be present in the input recognition area, to change.

First claim

The invention claimed is:

1. An information processing apparatus, comprising:
a first camera configured to capture an image of a plurality of objects in real space;
a storage unit configured to store a plurality of first feature amounts corresponding to the plurality of objects; and
circuitry configured to:
determine a plurality of second feature amounts corresponding to the plurality of objects, wherein the plurality of second feature amounts is determined based on the captured image;
recognize the plurality of objects in the real space based on the plurality of first feature amounts and the plurality of second feature amounts;
control, based on the recognition, a display screen to display a user interface, wherein
the user interface comprises a plurality of virtual objects,
a position of each virtual object of the plurality of virtual objects in the display screen corresponds to a position of a respective object of the recognized plurality of objects in the captured image, and
a posture of each virtual object of the plurality of virtual objects is changeable based on a posture of the respective object of the recognized plurality of objects;

determine, as a user gesture, a presence of a user object in an input recognition area of the first camera;
determine a region of interest within the user interface, wherein the region of interest corresponds to a virtual object of the plurality of virtual objects;
control, based on the determined region of interest, the display screen to display a specific object at a position corresponding to a position of an object of the recognized plurality of objects;
determine that the region of interest remains same during the presence of the user object in the input recognition area;
detect, as a termination of the user gesture, an absence of the user object in the input recognition area;
execute one of a tap operation, a click operation, or a select operation on the specific object based on
the determination that the region of interest remains same and
the detected absence of the user object in the input recognition area; and

control the display screen to display specific information on the specific object, wherein
the display of the specific information on the specific object is based on the detected absence of the user object in the input recognition area and the execution of the one of the tap operation, the click operation, or the selection operation, and
the specific information includes at least one of a title associated with the specific object or contents associated with the specific object.

2. The information processing apparatus according to claim 1 , further comprising a sensor configured to generate data representative of a user viewing direction, wherein
the circuitry is further configured to determine the region of interest within the user interface based on the generated data.

3. The information processing apparatus according to claim 2 , wherein the sensor is one of a second camera, an acceleration sensor, or a gyro sensor.

4. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to:
determine a change in the region of interest within the user interface based on the user gesture; and
execute a drag operation on the specific object based on the change of the region of interest.

5. The information processing apparatus according to claim 4 , wherein the circuitry is further configured to control, based on the change in the region of interest, a first device to transmit content information to a second device.

6. The information processing apparatus according to claim 5 , wherein the first device is a television.

7. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to transmit, based on the user gesture, a request to a first device to send determined data to the information processing apparatus.

8. The information processing apparatus according to claim 7 , wherein the circuitry is further configured to receive the determined data from the first device based on the request.

9. The information processing apparatus according to claim 7 , wherein the circuitry is further configured to receive a discount coupon from the first device based on the request.

10. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to:
determine a change in the region of interest within the user interface based on the user gesture; and
execute a drag operation on the specific object based on an amount of the change of the region of interest.

11. The information processing apparatus according to claim 1 , wherein the first camera is further configured to capture an image of the user object.

12. The information processing apparatus according to claim 1 , further comprising the display screen, wherein the display screen is in a head-mounted display unit.

13. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to determine, as the user gesture, a gesture of movement of a part of an operation body.

14. An information processing system, comprising:
a camera configured to capture an image of a plurality of objects in real space;
a storage unit configured to store a plurality of first feature amounts corresponding to the plurality of objects; and
circuitry configured to:
determine a plurality of second feature amounts corresponding to the plurality of objects, wherein the plurality of second feature amounts is determined based on the captured image;
recognize the plurality of objects in the real space based on the plurality of first feature amounts and the plurality of second feature amounts;
control, based on the recognition, a display screen to display a user interface, wherein
the user interface comprises a plurality of virtual objects,
a position of each virtual object of the plurality of virtual objects in the display screen corresponds to a position of a respective object of the recognized plurality of objects in the captured image, and
a posture of each virtual object of the plurality of virtual objects is changeable based on a posture of the respective object of the recognized plurality of objects;

determine, as a user gesture, a presence of a user object in an input recognition area of the camera;
determine a region of interest within the user interface, wherein the region of interest corresponds to a virtual object of the plurality of virtual objects;
control, based on the determined region of interest, the display screen to display a specific object at a position corresponding to a position of an object of the recognized plurality of objects;
determine that the region of interest remains same during the presence of the user object in the input recognition area;
detect, as a termination of the user gesture, an absence of the user object in the input recognition area;
execute one of a tap operation, a click operation, or a select operation on the specific object based on
the determination that the region of interest remains same and
the detected absence of the user object in the input recognition area; and

control the display screen to display specific information on the specific object, wherein
the display of the specific information on the specific object is based on the detected absence of the user object in the input recognition area and the execution of the one of the tap operation, the click operation, or the selection operation, and
the specific information includes at least one of a title associated with the specific object or contents as

Assignees

• Sony Corp

Inventors

• Kimura Kentaro
• Ikeda Takuo
• Ikeda Tetsuo
• Tsukahara Tsubasa
• Koga Yasuyuki
• Izumihara Atsushi

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06V40/107

  Static hand or arm · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title