Information processing device, information processing method, and program

Patent metadata

Field	Value
Publication number	US-11943519-B2
Application number	US-201917617728-A
Country	US
Kind code	B2
Filing date	Jul 1, 2019
Priority date	Jul 1, 2019
Publication date	Mar 26, 2024
Grant date	Mar 26, 2024

Abstract

Provided is an information processing device coupled to a device which a user operates with a hand of the user and on the surface of which a plurality of sensors that each output information associated with a spatial positional relation with the surface of the hand of the user are arranged, the information processing device being configured to receive output from each of the sensors included in the device and generate an image in which an image element having been drawn with a pixel value determined on the basis of a value represented by the output from each of the sensors is arranged at a position corresponding to a position at which each of the sensors is arranged, on a mapping space obtained by plane-unfolding a portion that constitutes the surface of the device and on which the sensors are arranged, Further, the information processing device is configured to, upon receipt of input of the generated image, estimate information associated with the position of each of joints of the hand of the user who operates the device and output the estimated information as coordinate information in a three-dimensional space whose origin is placed on a point that moves with the device.

First claim

The invention claimed is:

1. An information processing device coupled to a device that is operated by a user with a hand of the user and on a surface of which a plurality of sensors that each output information associated with a spatial positional relation with a surface of the hand of the user are arranged, the information processing device comprising:
processing circuitry configured to
receive the output from each of the sensors included in the device;
generate an image in which an image element having been drawn with a pixel value determined on a basis of a value represented by the output from each of the sensors is arranged at a position corresponding to a position at which each of the sensors is arranged, on a mapping space obtained by plane-unfolding at least a portion that constitutes the surface of the device and on which the sensors are arranged; and
upon receipt of input of the generated image, estimate information associated with a position of each of joints of the hand of the user who operates the device and that outputs the estimated information as coordinate information in a three-dimensional space whose origin is placed on a point that moves with the device,
wherein the processing circuitry for estimating the information associated with a position of each of the joints is further configured to use a result of machine learning that, in a same space as a space of the input image, estimates a position closest to each of the joints of the hand of the user who operates the device and a distance from the position to each of the joints.

2. The information processing device according to claim 1 , wherein the processing circuitry for estimating the information associated with the position of each of the joints is further configured to
use a result of the estimation made by using the result of the machine learning, and
convert the estimated information into coordinate information in the three-dimensional space whose origin is placed on the point that moves with the device, and
output the coordinate information.

3. The information processing device according to claim 1 , wherein the processing circuitry is further configured to
estimate the information associated with the position of each of the joints of the hand of the user who operates the device, by using a result of machine learning having machine-learned a relation between the input image and the information associated with the position of each of the joints of the hand of the user who operates the device, and
output the estimated information as coordinate information in the three-dimensional space whose origin is placed on the point that moves with the device.

4. The information processing device according to claim 1 , wherein the sensors arranged on the device are electrostatic capacitance sensors, and the received output from each of the sensors is a result of detection by each of the electrostatic capacitance sensors.

5. A control method for an information processing device coupled to a device that is operated by a user with a hand of the user and on a surface of which a plurality of sensors that each output information associated with a spatial positional relation with a surface of the hand of the user are arranged, the control method comprising:
by the information processing device,
receiving the output from each of the sensors included in the device;
generating an image in which an image element having been drawn with a pixel value determined on a basis of a value represented by the output from each of the sensors is arranged at a position corresponding to a position at which the each of the sensors is arranged, on a mapping space obtained by plane-unfolding at least a portion that constitutes the surface of the device and on which the sensors are arranged; and
estimating, upon receipt of input of the generated image, information associated with a position of each of joints of the hand of the user who operates the device and outputting the estimated information as coordinate information in a three-dimensional space whose origin is placed on a point that moves with the device,
wherein the processing circuitry for estimating the information associated with a position of each of the joints is further configured to use a result of machine learning that, in a same space as a space of the input image, estimates a position closest to each of the joints of the hand of the user who operates the device and a distance from the position to each of the joints.

6. A non-transitory computer-readable storage medium storing computer-readable instructions thereon which, when executed for a computer coupled to a device that is operated by a user with a hand of the user and on a surface of which a plurality of sensors that each output information associated with a spatial positional relation with a surface of the hand of the user are arranged, performs a method, the method comprising:
receiving the output from each of the sensors included in the device;
generating an image in which an image element having been drawn with a pixel value determined on a basis of a value represented by the output from each of the sensors is arranged at a position corresponding to a position at which the each of the sensors is arranged, on a mapping space obtained by plane-unfolding at least a portion that constitutes the surface of the device and on which the sensors are arranged; and
estimating, upon receipt of input of the generated image, information associated with a position of each of joints of the hand of the user who operates the device and outputting the estimated information as coordinate information in a three-dimensional space whose origin is placed on a point that moves with the device,
wherein the processing circuitry for estimating the information associated with a position of each of the joints is further configured to use a result of machine learning that, in a same space as a space of the input image, estimates a position closest to each of the joints of the hand of the user who operates the device and a distance from the position to each of the joints.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Arimatsu Kazuyuki
• Mori Hideki

Classifications

• H04N23/50 Primary

  Constructional details · CPC title

• G06F3/044

  by capacitive means · CPC title

• G06F3/014 Primary

  Hand-worn input/output arrangements, e.g. data gloves · CPC title