Calculation system, calculation method, and storage medium

Patent metadata

Field	Value
Publication number	US-11461923-B2
Application number	US-202017117631-A
Country	US
Kind code	B2
Filing date	Dec 10, 2020
Priority date	Dec 11, 2019
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

According to one embodiment, a calculation system includes a detector, a combiner, and a duration calculator. The detector refers to a plurality of images of an object imaged from mutually-different angles, detects the object in each of the images, and calculates a provisional position of the object in a coordinate system for each of the images. The coordinate system is prescribed. The combiner calculates a combined position of the object in the coordinate system by using the calculated plurality of provisional positions. The duration calculator refers to process information and a plurality of execution regions and calculates a duration of at least a portion of a plurality of processes based on the combined position and the execution regions. The process information includes information relating to the processes. The execution regions are represented using the coordinate system. The processes are executed in the execution regions.

First claim

What is claimed is:

1. A calculation system, comprising:
a detector referring to a plurality of images of a person imaged from mutually-different angles, detecting poses of the person in each of the plurality of images, calculating a provisional position of the person in a coordinate system for each of the plurality of images, and calculating a plurality of certainties respectively for the plurality of provisional positions by using the poses, the coordinate system being prescribed;
a combiner calculating a combined position of the person in the coordinate system by using the plurality of provisional positions and the plurality of certainties; and
a duration calculator referring to process information and a plurality of execution regions and calculating a duration of at least a portion of a plurality of processes based on the combined position and the plurality of execution regions, the process information including information relating to the plurality of processes, the execution regions being represented using the coordinate system, the processes being executed in the execution regions.

2. The calculation system according to claim 1 , further comprising:
an outputter outputting at least one of the combined position, one or more of the plurality of provisional positions, or one or more of the plurality of images.

3. The calculation system according to claim 2 , wherein
the outputter outputs a portion of the plurality of images based on the plurality of certainties.

4. The calculation system according to claim 2 , wherein
the outputter also outputs a change of the combined position over time.

5. The calculation system according to claim 1 , wherein the detector identifies the person.

6. The calculation system according to claim 5 , wherein
the detector calculates each of the plurality of certainties by using at least one of
a confidence level of an identification result of the person,
a size of the person in the image,
a distance between the person and a center of the image,
a value indicating the pose of the person, or
an exposure degree of the person in the image.

7. The calculation system according to claim 1 , wherein
the detector
calculates the plurality of provisional positions in the coordinate system, the coordinate system being two-dimensional, and
corrects the plurality of provisional positions by using an elevation difference between a first region and a floor surface when the detected person is included in the first region, the first region being higher than the floor surface.

8. The calculation system according to claim 1 , wherein
when a grounded part of the person on a floor surface is not visible in the image, the detector estimates a position of the grounded part of the person of the image and calculates the provisional position based on a result of the estimation.

9. The calculation system according to claim 1 , wherein
the detector
detects the person and identifies the person for the plurality of images, and
calculates the provisional position for a plurality of persons when the plurality of persons is visible in the plurality of images, and p 1 the combiner calculates the combined position for each of the plurality of persons.

10. The calculation system according to claim 1 , wherein
the process information includes times that the plurality of processes are executed, and
the duration calculator compares the combined positions at the times to the execution regions of the processes executed at the times and calculates durations that the combined positions are included in the execution regions as the durations of the at least a portion of the plurality of processes.

11. The calculation system according to claim 1 , further comprising:
a plurality of imaging devices imaging the person from mutually-different angles.

12. A calculation method, comprising:
referring to, by processing circuitry, a plurality of images of a person imaged from mutually-different angles;
detecting, by the processing circuitry, poses of the person for each of the plurality of images;
calculating, by the processing circuitry, provisional positions of the person in a coordinate system, the coordinate system being prescribed;
calculating, by the processing circuitry, a plurality of certainties respectively for the plurality of provisional positions by using the poses;
calculating, by the processing circuitry, a combined position of the person in the coordinate system by using the plurality of provisional positions and the plurality of certainties;
referring to, by the processing circuitry,
process information including information relating to a plurality of processes, and
a plurality of execution regions represented using the coordinate system, the processes being executed in the execution regions; and

calculating, by the processing circuitry, a duration of at least a portion of the plurality of processes based on the combined position and the plurality of execution regions.

13. The method according to claim 12 , wherein
the person is detected,
the person is identified, and
each of the plurality of certainties is calculated by using at least one of
a confidence level of an identification result of the person,
a size of the person in the image,
a distance between the person and a center of the image,
a value indicating the pose of the person, or
an exposure degree of the person in the image.

14. A non-transitory computer-readable storage medium storing a program,
the program causing a computer to
refer to a plurality of images of a person imaged from mutually- different angles,
detect poses of the person for each of the plurality of images,
calculate provisional positions of the person in a coordinate system, the coordinate system being prescribed,
calculate a plurality of certainties respectively for the plurality of provisional positions by using the poses,
calculate a combined position of the person in the coordinate system by using the plurality of provisional positions and the plurality of certainties,
refer to
process information including information relating to a plurality of processes, and
a plurality of execution regions represented using the coordinate system, the processes being executed in the execution region, and

calculate a duration of at least a portion of the plurality of processes based on the combined position and the plurality of execution regions.

15. The non-transitory computer-readable storage medium according to claim 14 , wherein
the program causes the computer to
detect the person and identify the person, and
calculate each of the plurality of certainties by using at least one of
a confidence level of an identification result of the person,
a size of the person in the image,
a distance between the person and a center of the image,
a value indicating the pose of the person, or
an exposure degree of the person in the image.

Assignees

• Toshiba Kk

Inventors

• Aoki Yusuke
• Tonouchi Yojiro
• Nakasu Toshiaki
• Shibata Tomoyuki
• Sakurai Yuki
• Kase Akiko
• Morinaga Ryo
• Iwasaki Masaki
• Kawahata Sho

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title