Information processing apparatus and information processing method

Patent metadata

Field	Value
Publication number	US-12169929-B2
Application number	US-202017430869-A
Country	US
Kind code	B2
Filing date	Jan 10, 2020
Priority date	Feb 22, 2019
Publication date	Dec 17, 2024
Grant date	Dec 17, 2024

Abstract

A configuration that analyzes reflection light of light emitted to a user face and executes both facial-expression analysis and biometric-signal analysis together is realized. The configuration has a light-receiving section that receives reflection light of light emitted to a user face, and a light-reception-signal analyzing section that analyzes a light-reception signal of the light-receiving section. The light-reception-signal analyzing section has a facial-expression analyzing section that analyzes user-skin-surface reflection light and generates facial-expression analysis information, and a biometric-signal analyzing section that analyzes subepidermal reflection light and generates bioanalysis information. The light-reception signal of the light-receiving section includes skin-surface reflection light and subepidermal-tissue reflection light, and the facial-expression analyzing section extracts a low frequency component from the light-reception signal, acquires the skin-surface reflection light, and executes facial-expression analysis. The biometric-signal analyzing section extracts a high frequency component from the light-reception signal, acquires the subepidermal-tissue reflection light, and executes a biometric-signal analysis process.

First claim

The invention claimed is:

1. An information processing apparatus, comprising:
a light-emitting section configured to emit light to a user face;
a light-receiving section configured to:
receive first reflection light based on the light emitted to the user face; and
output a light-reception signal based on the received first reflection light; and

a light-reception-signal analyzing section configured to analyze the light-reception signal of the light-receiving section, wherein
the light-reception-signal analyzing section includes:
a facial-expression analyzing section configured to generate facial-expression analysis information based on the first reflection light; and
a biometric-signal analyzing section configured to:
extract a high frequency component from the light-reception signal of the light-receiving section;
acquire subepidermal-tissue reflection light based on the high frequency component;
execute a biometric-signal analysis process based on the subepidermal-tissue reflection light; and
generate first bioanalysis information based on the biometric-signal analysis process.

2. The information processing apparatus according to claim 1 , wherein
the light-reception signal of the light-receiving section includes skin-surface reflection light and the subepidermal-tissue reflection light,
the facial-expression analyzing section is further configured to:
extract a low frequency component from the light-reception signal of the light-receiving section,
acquire the skin-surface reflection light based on the extracted low frequency component, and
execute facial-expression analysis process based on the acquired skin-surface reflection light.

3. The information processing apparatus according to claim 1 , wherein
the facial-expression analyzing section includes a facial-expression analysis information storage section,
the facial-expression analysis information storage section is configured to store a learning model, and
the facial-expression analyzing section is further configured to execute a facial-expression analysis process based on the stored learning model.

4. The information processing apparatus according to claim 3 , wherein
the learning model includes corresponding data of facial-expression labels and feature data of skin-surface reflection-light signals,
the skin-surface reflection-light signals are associated with skin-surface movements, and
the facial-expression labels correspond to a plurality of different facial expressions.

5. The information processing apparatus according to claim 1 , wherein
the biometric-signal analyzing section includes a biometric-signal analysis information storage section,
the biometric-signal analysis information storage section is configured to store a learning model, and
the biometric-signal analyzing section is further configured to execute a bioanalysis process based on the stored learning model.

6. The information processing apparatus according to claim 5 , wherein
the learning model includes corresponding data of feature data of subepidermal reflection-light signals,
the subepidermal reflection-light signals correspond to a plurality of different biological states, and
the feature data of the subepidermal reflection-light signals reflect subepidermal states.

7. The information processing apparatus according to claim 1 , wherein the generated first bioanalysis information includes least one of a blood flow rate, a blood flow speed, a pulse wave, a blood pressure, a heart rate, a heart rate fluctuation, a blood oxygen concentration, or arterial oxygen saturation.

8. The information processing apparatus according to claim 1 , wherein
the light-receiving section includes a plurality of sensors, and
each of the plurality of sensors is attached at a respective position of a plurality of positions of a head-mounted display.

9. The information processing apparatus according to claim 1 , wherein the light-reception-signal analyzing section is further configured to execute analysis of second reflection light from a plurality of different positions on the user face.

10. The information processing apparatus according to claim 1 , wherein
the light-reception-signal analyzing section is further configured to:
execute analysis of second reflection light from a plurality of different positions on the user face; and
acquire different pieces of second bioanalysis information corresponding to the plurality of different positions.

11. The information processing apparatus according to claim 1 , wherein
the light-reception-signal analyzing section is further configured to:
execute analysis of second reflection light from a plurality of different positions on the user face; and
acquire second bioanalysis information by complementarily using information acquired from the plurality of different positions.

12. The information processing apparatus according to claim 1 , further comprising:
a high-precision affect analyzing section configured to:
receive a first input and a second input, wherein the first input includes the facial-expression analysis information and the second input includes the first bioanalysis information; and
generate high-precision affect information based on the facial-expression analysis information and the first bioanalysis information.

13. The information processing apparatus according to claim 12 , wherein
the high-precision affect analyzing section includes a high-precision affect analysis information storage section,
the high-precision affect analysis information storage section is configured to store a learning model, and
the high-precision affect analyzing section is further configured to execute an affect analysis process based on the stored learning model.

14. The information processing apparatus according to claim 13 , wherein
the learning model is a model in which feature data that corresponds to a plurality of different affect states is associated, and
the feature data includes feature data of facial-expression information and feature data of bioinformation.

15. The information processing apparatus according to claim 12 , further comprising a game control section configured to execute game control based on the high-precision affect information.

16. The information processing apparatus according to claim 12 , further comprising a facial-expression authenticity determining section configured to execute facial-expression authenticity determination based on the high-precision affect information.

17. The information processing apparatus according to claim 1 , further comprising an avatar control section configured to execute avatar control based on the facial-expression analysis information and the first bioanalysis information.

18. An information processing method, comprising:
emitting, by a light-emitting section, light to a user face;
receiving, by a light-receiving section, reflection light based on the light emitted to the user face;
outputting, by the light-receiving section, a light-reception signal based on the received reflection light;
analyzing, by a light-reception-signal analyzing section, the light-reception signal of the light-receiving section, wherein the light-reception-signal analyzing section includes a facial-expression analyzing section and a biometric-signal analyzing section;
executing, by the facial-expression analyzing section, a facial-expression analysis process to generate facial-expression analysis information, wherein the facial-expression analysis information is generated based on the reflection l

Assignees

• Sony Group Corp

Inventors

• Ogawa Sayaka
• Aizawa Kota

Classifications

• A63F2300/1087

  comprising photodetecting means, e.g. a camera · CPC title

• G06V40/168

  Feature extraction; Face representation · CPC title

• G06V40/174

  Facial expression recognition · CPC title

• A63F13/213 Primary

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• G06V40/166

  using acquisition arrangements · CPC title