Moving image integration device, moving image integration method, and moving image integration program

Patent metadata

Field	Value
Publication number	US-2025182482-A1
Application number	US-202318842909-A
Country	US
Kind code	A1
Filing date	Feb 1, 2023
Priority date	Mar 11, 2022
Publication date	Jun 5, 2025
Grant date	—

Abstract

The present disclosure provides a video image integrating device including an acquiring section acquiring video images in which work of a worker is captured; a detecting section; a judging section; and a generating section.

First claim

1 . A video image integrating device comprising:
a memory; and at least one processor coupled to the memory, the at least one processor being configured to: acquire video images in which work of a worker is captured; based on the video images, detect time-series data of detection information relating to a skeleton or a region of the worker; based on the detected time-series data of the detection information, and for each of a plurality of types of scenarios to be extracted, judge whether the work is work satisfying a condition corresponding to the scenario to be extracted; and based on a number of times of extraction or an extraction time period that were set for each of the plurality of types of scenarios to be extracted, generate video images that integrate video images of a portion that includes a point in time when it was judged that the work is work satisfying a condition corresponding to the scenario to be extracted.

2 . The video image integrating device of claim 1 , wherein:
the scenarios to be extracted include scenarios in which a time period of a work cycle is greater than or equal to a threshold value, and for a scenario in which the time period of the work cycle is greater than or equal to the threshold value, based on the detected time-series data of the detection information, the at least one processor analyzes a time period of a work cycle each work cycle, and, if the time period of a work cycle is greater than or equal to the threshold value, judges that the work is work satisfying the condition.

3 . The video image integrating device of claim 1 , wherein:
the scenarios to be extracted include scenarios in which a worker caries out a specific action, and for a scenario in which the worker carries out a specific action, based on the detected time-series data of the detection information, the at least one processor judges that the work is work satisfying the condition in a case in which the worker has moved to a position corresponding to a place where the specific action is carried out.

4 . The video image integrating device of claim 3 , wherein the specific action is placing a defective product at a defective product place.

5 . The video image integrating device of claim 1 , wherein:
the scenarios to be extracted include scenarios in which an error log relating to a device used in the work arises, and for a scenario in which an error log arises, in a case in which a log relating to the device used in the work is an error log, the at least one processor further judges that the work is work satisfying the condition.

6 . A video image integrating method comprising:
an acquiring section acquiring video images in which work of a worker is captured; based on the video images, a detecting section detecting time-series data of detection information relating to a skeleton or a region of the worker; based on the detected time-series data of the detection information, and for each of a plurality of types of scenarios to be extracted, a judging section judging whether the work is work satisfying a condition corresponding to the scenario to be extracted; and based on a number of times of extraction or an extraction time period that were set for each of the plurality of types of scenarios to be extracted, a generating section generating video images that integrate video images of a portion that includes a point in time when it was judged that the work is work satisfying a condition corresponding to the scenario to be extracted.

7 . A non-transitory storage medium storing a program executable by a computer so as to execute video image integrating processing, the video image integrating processing including:
acquiring video images in which work of a worker is captured; based on the video images, detecting time-series data of detection information relating to a skeleton or a region of the worker; based on the detected time-series data of the detection information, and for each of a plurality of types of scenarios to be extracted, judging whether the work is work satisfying a condition corresponding to the scenario to be extracted; and based on a number of times of extraction or an extraction time period that were set for each of the plurality of types of scenarios to be extracted, generating video images that integrate video images of a portion that includes a point in time when it was judged that the work is work satisfying a condition corresponding to the scenario to be extracted.

8 . The video image integrating device of claim 2 , wherein:
the scenarios to be extracted include scenarios in which a worker caries out a specific action, and for a scenario in which the worker carries out a specific action, based on the detected time-series data of the detection information, the at least one processor judges that the work is work satisfying the condition in a case in which the worker has moved to a position corresponding to a place where the specific action is carried out.

9 . The video image integrating device of claim 8 , wherein the specific action is placing a defective product at a defective product place.

10 . The video image integrating device of claim 2 , wherein:
the scenarios to be extracted include scenarios in which an error log relating to a device used in the work arises, and for a scenario in which an error log arises, in a case in which a log relating to the device used in the work is an error log, the at least one processor further judges that the work is work satisfying the condition.

Assignees

• Omron Tateisi Electronics Co

Inventors

• Miyazaki Masashi
• Wada Hirotaka
• Onda Hiroomi
• Nishiyuki Kenta

Classifications

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title