Data analysis and training data, generation device, system, method, and generation program

Patent metadata

Field	Value
Publication number	US-12488258-B2
Application number	US-202017779367-A
Country	US
Kind code	B2
Filing date	Oct 9, 2020
Priority date	Nov 28, 2019
Publication date	Dec 2, 2025
Grant date	Dec 2, 2025

Abstract

A generation device executes an inference process of inputting analysis target data to a plurality of inference models and outputting a plurality of inference results related to an object included in the analysis target data from the plurality of inference models, a determination process of determining, based on the plurality of inference results, a specific inference result from the plurality of inference results output by the inference process, and a generation process of generating a training data set including the specific inference result determined by the determination process and the analysis target data.

First claim

The invention claimed is:

1 . A generation device, comprising:
a processor configured to execute a program; and a storage device configured to store the program, wherein the processor is configured to execute; an inference process of inputting analysis target data, which is image data including a plurality of time-series frames, to a plurality of inference models and outputting a plurality of inference results related to an object included in the analysis target data from the plurality of inference models; a determination process of determining, based on a movement amount of a skeleton point of a skeleton of the object, between a first frame and a second frame in each of the plurality of inference results, wherein the skeleton indicates a shape of the object, one of the plurality of inference results output by the inference process, for which the movement amount is below a predetermined threshold, as a specific inference result; and a generation process of generating a training data set including the specific inference result determined by the determination process and the analysis target data.

2 . The generation device according to claim 1 , wherein
in the determination process, the processor determines the specific inference result from the plurality of inference results based on a movement amount of the object between a first frame and a second frame in each of the plurality of inference results.

3 . The generation device according to claim 1 , wherein
in the determination process, the processor determines the specific inference result from the plurality of inference results based on inference accuracy of the object included in each of the plurality of inference results.

4 . The generation device according to claim 1 , wherein
in the determination process, the processor determines the specific inference result from the plurality of inference results based on inference accuracy of the skeleton point of the skeleton of the object, included in each of the plurality of inference results.

5 . The generation device according to claim 1 , wherein
in the determination process, when a classification of the object included in each of the plurality of inference results is different, the processor excludes an inference result including a classification of a minority from the plurality of inference results.

6 . The generation device according to claim 1 , wherein
the processor is configured to execute; a determination process of determining a specific inference model from the plurality of inference models based on the number of outputs of the specific inference result in each of the inference models.

7 . The generation device according to claim 6 , wherein
the processor is configured to execute; a learning process of training the specific inference model, which is determined by the determination process, by using the training data set generated by the generation process, and outputting the specific inference model as a learning model.

8 . The generation device according to claim 7 , wherein the processor is configured to execute:
a judgment process of judging whether inference accuracy included in a learning result, which is obtained as a result of inputting test data that is not input to the plurality of inference models to the learning model output by the learning process, is equal to or higher than predetermined inference accuracy.

9 . The generation device according to claim 8 , wherein
in the judgment process, when the inference accuracy included in the learning result is equal to or higher than the predetermined inference accuracy, the processor outputs the learning model.

10 . The generation device according to claim 8 , wherein
in the determination process, when it is judged by the judgment process that the inference accuracy included in the learning result is not equal to or higher than the predetermined inference accuracy, the processor determines the specific inference result, from a remaining inference result that is not determined as the specific inference result among the plurality of inference results, based on the remaining inference result.

11 . A data analysis system, comprising:
a generation device, including a processor configured to execute a program and a storage device configured to store the program, the generation device being configured to generate a learning model; and an edge system, including a plurality of edge devices, each of which includes a sensor, and an edge management device that manages the plurality of edge devices, the edge system being configured to acquire analysis target data, wherein the generation device is configured to execute: a first inference process of inputting first analysis target data, which is image data including a plurality of time-series frames, to a plurality of inference models and outputting a plurality of inference results related to an object included in the first analysis target data from the plurality of inference models; a determination process of determining, based on a movement amount of a skeleton point of a skeleton of the object, between a first frame and a second frame in each of the plurality of inference results, wherein the skeleton indicates a shape of the object, one of the plurality of inference results output by the first inference process, for which the movement amount is below a predetermined threshold, as a specific inference result; a generation process of generating a training data set including the specific inference result determined by the determination process and the analysis target data; a determination process of determining a specific inference model from the plurality of inference models based on the number of outputs of the specific inference result in each of the inference models; and a learning process of training the specific inference model, which is determined by the determination process, by using the training data set generated by the generation process, and outputting the specific inference model as a learning model; and the edge system is configured to execute: an acquisition process of acquiring the first analysis target data and second analysis target data from an analysis environment; an output process of outputting the first analysis target data acquired by the acquisition process; and a second inference process of inputting the second analysis target data acquired by the acquisition process to the learning model output by the learning process and outputting an inference result related to an object included in the second analysis target data from the learning model.

12 . A generation method to be executed by a generation device including a processor configured to execute a program, and a storage device configured to store the program, the generation method comprising:
the processor executing: an inference process of inputting analysis target data, which is image data including a plurality of time-series frames, to a plurality of inference models and outputting a plurality of inference results related to an object included in the analysis target data from the plurality of inference models; a determination process of determining, based on a movement amount of a skeleton point of a skeleton of the object, between a first frame and a second frame in each of the plurality of inference results, wherein the skeleton indicates a shape of the object, one of the plurality of inference results output by the inference process, for which the movement amount is below a predetermined threshold, as a specific inference result; and a generation process of generating a training data set including the specific inference result determi

Assignees

• Hitachi Ltd

Inventors

• Neo Atsushi

Classifications

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title