Information processing apparatus and information processing method

Patent metadata

Field	Value
Publication number	US-11216179-B2
Application number	US-201916969515-A
Country	US
Kind code	B2
Filing date	Jan 23, 2019
Priority date	Feb 20, 2018
Publication date	Jan 4, 2022
Grant date	Jan 4, 2022

Abstract

Provided is an information processing apparatus including a processing unit that determines an angle between at least a part of an upper limb of a user and one of a vector or a plane. The processing unit further determines whether an operation of the user is a pointing operation or a hover operation on the basis of the angle that is between at least the part of the upper limb of the user and one of the vector or the plane.

First claim

The invention claimed is:

1. An information processing apparatus, comprising:
circuitry configured to:
acquire input information associated with a user, wherein the acquired input information comprises an operation of the user;
determine, based on the input information, an angle between at least a part of an upper limb of the user and one of a first vector or a specific plane, wherein each of the first vector and the specific plane corresponds to a surface of an object indicated by the operation of the user;
determine that the operation of the user is a pointing operation based on the angle that is larger than a threshold angle;
determine that the operation of the user is a hover operation based on the angle that is smaller than the threshold angle; and
control, based on the operation that is one of the pointing operation or the hover operation, execution of a specific process corresponding to the object.

2. The information processing apparatus according to claim 1 , wherein at least the part of the upper limb is at least a part of a hand of the user.

3. The information processing apparatus according to claim 2 , wherein
at least the part of the hand corresponds to a second vector, and
the second vector is based on a specific position of the hand and a tip position of a first finger of the user.

4. The information processing apparatus according to claim 3 ,
wherein the specific position of the hand is one of a center position of the hand or a joint position of the first finger.

5. The information processing apparatus according to claim 3 ,
wherein the specific position of the hand is a joint position of the first finger.

6. The information processing apparatus according to claim 5 ,
wherein each of the first vector and the specific plane is based on the joint position of the first finger and a center position of the hand.

7. The information processing apparatus according to claim 5 ,
wherein each of the first vector and the specific plane is based on a joint position of a second finger and a tip position of the second finger.

8. The information processing apparatus according to claim 1 ,
wherein at least the part of the upper limb is at least a part of an arm of the user.

9. The information processing apparatus according to claim 8 , wherein
at least the part of the arm corresponds to a second vector, and
the second vector is based on a wrist position of the user and an elbow joint position of the user.

10. The information processing apparatus according to claim 8 ,
wherein each of the first vector or the specific plane is based on an upper arm position of the user and an elbow joint position of the user.

11. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to control a projector to project the object indicated by the operation of the user.

12. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to:
control execution of a first process based on the operation of the user is the pointing operation; and
control execution of a second process different from the first process based on the operation of the user is the hover operation.

13. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to execute a system program and an application program, and
the system program notifies the application program of an indication position based on the operation of the user is one of the pointing operation or the hover operation.

14. The information processing apparatus according to claim 13 ,
wherein based on the operation of the user is the hover operation, the system program notifies the application program of two-dimensional coordinates of a tip of a first finger of the user in space.

15. The information processing apparatus according to claim 13 ,
wherein based on the operation of the user is the pointing operation, the system program notifies the application program of coordinates on a display surface, indicated by an extension of a tip position of a first finger of the user.

16. The information processing apparatus according to claim 13 ,
wherein the system program notifies the application program of a gesture type operation that indicates the operation of the user is one of the pointing operation or the hover operation.

17. The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to execute a system program and an application program,
the application program designates a gesture type determination condition that indicates the operation of the user is one of the pointing operation or the hover operation, and
the system program issues, to the application program, a notification of a gesture type operation based on the designation of the gesture type determination condition.

18. The information processing apparatus according to claim 17 , wherein
the application program designates, as the gesture type determination condition, a weight assigned to at least one parameter, and
the application program determines the gesture type operation based on the designation of the weight assigned to the at least one parameter.

19. The information processing apparatus according to claim 18 ,
wherein based on the designation of the weight assigned to the at least one parameter as the gesture type determination condition, the application program sets the weight assigned to each of a plurality of parameters to be mutually different.

20. An information processing method, comprising:
acquiring input information associated with a user, wherein the acquired input information comprises an operation of the user;
determining, based on the input information, an angle between at least a part of an upper limb of the user and one of a specific vector or a specific plane, wherein each of the specific vector and the specific plane corresponds to a surface of an object indicated by the operation of the user;
determining that the operation of the user is a pointing operation based on the angle that is larger than a threshold angle;
determining that the operation of the user is a hover operation based on the angle that is smaller than the threshold angle; and
controlling, based on the operation that is one of the pointing operation or the hover operation, execution of a specific process corresponding to the object.

21. A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by circuitry of an information processing apparatus, cause the circuitry to execute operations, the operations comprising:
acquiring input information associated with a user, wherein the acquired input information comprises an operation of the user;
determining, based on the input information, an angle between at least a part of an upper limb of the user and one of a specific vector or a specific plane, wherein each of the specific vector and the specific plane corresponds to a surface of an object indicated by the operation of the user;
determining that the operation of the user is a pointing operation based on the angle that is larger than a threshold angle;
determining that the operation of the user is a hover operation based on the angle that is smaller than the threshold angle; and
controlling, based on the operation that is one of the pointing operation or the hover operation, execution of a specific process corresponding to the object.

Assignees

• Sony Corp

Inventors

• Ikeda Tetsuo
• Fujinawa Eisuke
• Yoshitomi Kosuke
• Handa Masaki
• Otsuka Junji
• Gotoh Kenji
• Miyazawa Katsuji
• Watanabe Shoji

Classifications

• G06F3/04845

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• G06F3/0425

  using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386) · CPC title

• G06F3/0354

  with detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks · CPC title