Information processing system and information processing method

Patent metadata

Field	Value
Publication number	US-12087090-B2
Application number	US-202217688790-A
Country	US
Kind code	B2
Filing date	Mar 7, 2022
Priority date	Sep 30, 2019
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

Provided are an information processing system and an information processing method which can easily grasp a state of a person in a predetermined area. The information processing system includes a first imaging unit that images a performer, a second imaging unit that images a person in a predetermined area, a first recognition unit that recognizes a scene based on an image captured by the first imaging unit, a second recognition unit that recognizes expression of the person based on an image captured by the second imaging unit, a calculation unit that calculates a state index of the person according to the scene based on a recognition result of the scene and a recognition result of the expression of the person, a heat map creation unit that creates a heat map representing the state index of the person in association with a position of the person in the predetermined area, and an output unit that outputs the heat map.

First claim

What is claimed is:

1. An information processing system comprising:
a processor; and
a memory connected to the processor,
wherein the processor is configured to
perform first recognition in which a scene is recognized based on an image obtained by imaging a performer,
perform second recognition in which expression of a person in a predetermined area is recognized based on an image obtained by imaging the person,
calculate a state index of the person according to the scene based on a recognition result of the scene and a recognition result of the expression of the person,
create a heat map representing the state index of the person in association with a position of the person in the predetermined area, and
output the heat map.

2. The information processing system according to claim 1 ,
wherein the processor is further configured to
set a number of clusters based on a transmission ability of the performer,
perform clustering on the heat map according to the set number of clusters, and
in the output, output the clustered heat map.

3. The information processing system according to claim 2 ,
wherein the processor is further configured to, in the clustering, perform clustering on the heat map for data of the person having the state index equal to or more than a threshold value.

4. The information processing system according to claim 2 ,
wherein the processor is further configured to estimate the transmission ability of the performer based on the image obtained by imaging the performer.

5. The information processing system according to claim 2 ,
wherein the transmission ability includes at least one of skill or remaining power of the performer.

6. The information processing system according to claim 1 ,
wherein the processor is further configured to transform an image of the heat map into an image of a viewpoint different from the image obtained by imaging the person in the predetermined area.

7. The information processing system according to claim 1 ,
wherein the processor is further configured to invert color or shading of the heat map.

8. The information processing system according to claim 1 , further comprising:
a display that displays the heat map output from the processor.

9. The information processing system according to claim 1 , further comprising:
a projector that projects the heat map output from the processor onto the predetermined area.

10. The information processing system according to claim 9 ,
wherein the projector projects the heat map onto the predetermined area by projection mapping.

11. The information processing system according to claim 1 , further comprising:
a first camera that images the performer; and
a second camera that images the person in the predetermined area.

12. An information processing method comprising:
imaging a performer;
imaging a person in a predetermined area;
recognizing a scene based on an image obtained by imaging the performer;
recognizing expression of the person in the predetermined area based on an image obtained by imaging the person;
calculating a state index of the person according to the scene based on a recognition result of the scene and a recognition result of the expression of the person;
creating a heat map representing the state index of the person in association with a position of the person in the predetermined area; and
outputting the heat map.

13. The information processing method according to claim 12 , further comprising:
setting a number of clusters based on information on a transmission ability of the performer; and
performing clustering on the heat map according to the set number of clusters,
wherein, in the output of the heat map, the clustered heat map is output.

Assignees

• Fujifilm Corp

Inventors

• Hayashi Daisuke
• Makino Kenji
• Terada Masahiro
• Ego Shunta

Classifications

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06V10/762

  using clustering, e.g. of similar faces in social networks · CPC title

• G06V20/35

  Categorising the entire scene, e.g. birthday party or wedding scene · CPC title

• G06T7/00

  Image analysis · CPC title