Work recognition device, work recognition method, and recording medium storing work recognition program

Patent metadata

Field	Value
Publication number	US-2025239104-A1
Application number	US-202218704424-A
Country	US
Kind code	A1
Filing date	Nov 11, 2022
Priority date	Nov 18, 2021
Publication date	Jul 24, 2025
Grant date	—

Abstract

The present disclosure provides a work recognition device acquiring a photographed image capturing work of a worker; and detecting, based on the photographed image, first detection information relating to an object of the work and at least one of a right hand or a left hand of the worker.

First claim

1 . A work recognition device, comprising:
an acquisition section that acquires a photographed image capturing work of a worker; a first detection section that detects, based on the photographed image, first detection information relating to at least one of an object of the work or a hand of the worker, the hand being at least one of a right hand or a left hand of the worker; a second detection section that detects, based on the photographed image, second detection information relating to a skeleton of the worker; a first recognition section that executes first recognition processing that recognizes the work, based on the first detection information and the second detection information which have been detected; a second recognition section that executes second recognition processing that recognizes the work, based on the second detection information that has been detected; a determination section that determines, based on the first detection information, whether or not a switching condition for switching between recognizing the work by the first recognition section or recognizing the work by the second recognition section is satisfied; and an output section that, in a case in which the switching condition is not satisfied, outputs a recognition result of the work by the first recognition section, and in a case in which the switching condition is satisfied, outputs a recognition result of the work by the second recognition section.

2 . The work recognition device according to claim 1 , wherein:
the determination section determines that the switching condition is satisfied in a case in which a distance between the hand and the object is greater than or equal to a predetermined threshold value.

3 . The work recognition device according to claim 1 , wherein:
the determination section determines that the switching condition is satisfied in a case in which a distance between the right hand and the left hand is greater than or equal to a predetermined threshold value.

4 . The work recognition device according to claim 1 , wherein:
the determination section determines that the switching condition is satisfied in a case in which a bounding box of the hand and a bounding box of the object do not overlap.

5 . The work recognition device according to claim 1 , wherein:
the determination section determines whether or not the switching condition is satisfied based on a size of the object that has been detected.

6 . The work recognition device according to claim 5 , wherein:
the determination section determines that the switching condition is satisfied in a case in which a size of a bounding box of the object that has been detected is smaller than a predetermined minimum size of the object.

7 . The work recognition device according to claim 5 , wherein:
the determination section determines that the switching condition is satisfied in a case in which a size of a bounding box of the object that has been detected is larger than a predetermined maximum size of the object.

8 . The work recognition device according to claim 1 , wherein:
the first detection section calculates a degree of reliability of the object that has been detected; and the determination section determines that the switching condition is satisfied in a case in which the degree of reliability of the object is less than or equal to a predetermined threshold value.

9 . The work recognition device according to claim 1 , wherein:
the determination section determines that the switching condition is satisfied in a case in which a speed of at least one of the hand or the object, which have been detected, is greater than or equal to a predetermined threshold value.

10 . The work recognition device according to claim 1 , wherein:
the determination section determines that the switching condition is satisfied in a case in which a position of the hand that has been detected is outside a range of a predetermined work space.

11 . A work recognition method in which a computer executes processing, the processing comprising:
acquiring a photographed image capturing work of a worker; detecting, based on the photographed image, first detection information relating to at least one of an object of the work or a hand of the worker, the hand being at least one of a right hand or a left hand of the worker; detecting, based on the photographed image, second detection information relating to a skeleton of the worker; executing first recognition processing that recognizes the work, based on the first detection information and the second detection information which have been detected; executing second recognition processing that recognizes the work, based on the second detection information that has been detected; determining, based on the first detection information, whether or not a switching condition for switching between recognizing the work by the first recognition processing or recognizing the work by the second recognition processing is satisfied; and in a case in which the switching condition is not satisfied, outputting a recognition result of the work by the first recognition processing, and in a case in which the switching condition is satisfied, outputting a recognition result of the work by the second recognition processing.

12 . A non-transitory recording medium storing a work recognition program that causes a computer to execute processing, the processing comprising:
acquiring a photographed image capturing work of a worker; detecting, based on the photographed image, first detection information relating to at least one of an object of the work or a hand of the worker, the hand being at least one of a right hand or a left hand of the worker; detecting, based on the photographed image, second detection information relating to a skeleton of the worker; executing first recognition processing that recognizes the work, based on the first detection information and the second detection information which have been detected; executing second recognition processing that recognizes the work, based on the second detection information that has been detected; determining, based on the first detection information, whether or not a switching condition for switching between recognizing the work by the first recognition processing or recognizing the work by the second recognition processing is satisfied; and in a case in which the switching condition is not satisfied, outputting a recognition result of the work by the first recognition processing, and in a case in which the switching condition is satisfied, outputting a recognition result of the work by the second recognition processing.

Assignees

• Omron Tateisi Electronics Co

Inventors

• Miyazaki Masashi
• Nishiyuki Kenta
• Wada Hirotaka
• Kinoshita Koichi

Classifications

• G06V10/87

  using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V40/28 Primary

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V40/23

  Recognition of whole body movements, e.g. for sport training · CPC title