Processing system, processing method, and storage medium

Patent metadata

Field	Value
Publication number	US-2023245338-A1
Application number	US-202318157355-A
Country	US
Kind code	A1
Filing date	Jan 20, 2023
Priority date	Jan 31, 2022
Publication date	Aug 3, 2023
Grant date	—

Abstract

According to one embodiment, a processing system estimates a pose of a worker, a position of an article, an orientation of the article, and a state of the article based on an image of the worker and the article. The processing system further estimates a work spot of the worker on the article based on an estimation result of the pose, an estimation result of the position, and an estimation result of the orientation. The processing system further estimates a task performed by the worker based on an estimation result of the work spot and an estimation result of the state.

First claim

What is claimed is:

1 . A processing system,
the system
estimates a pose of a worker, a position of an article, an orientation of the article, and a state of the article based on an image of the worker and the article,
estimates a work spot of the worker on the article based on an estimation result of the pose, an estimation result of the position, and an estimation result of the orientation, and
estimates a task performed by the worker based on an estimation result of the work spot and an estimation result of the state.

2 . The system according to claim 1 , wherein
the estimation of the position and the orientation includes:
extracting a partial region from the image, the partial region including the worker and the article;
specifying a worker region based on the estimation result of the pose, the worker being imaged in the worker region;
estimating an article region by removing the worker region from the partial region, the article being imaged in the article region; and
estimating the position and the orientation by using the article region.

3 . The system according to claim 2 , wherein
the estimation of the position and the orientation includes:
estimating an outer edge of the article by using the article region; and
estimating the position and the orientation by using at least a part of the outer edge.

4 . The system according to claim 3 , wherein
the system extracts, as the partial region, a region of movement between a plurality of the images.

5 . The system according to claim 3 , wherein
the system:
extracts a plurality of the partial regions between the plurality of images;
estimates a plurality of the outer edges by using the plurality of partial regions; and
estimates the position and the orientation by using one of the plurality of outer edges.

6 . The system according to claim 1 , wherein
the system cuts out a part of the image including the article, and estimates the state by using the cut-out part of the image.

7 . The system according to claim 1 , wherein
the estimation of the work spot includes:
setting a plurality of gates in the article based on the estimation result of the position and the estimation result of the orientation; and
estimating the work spot based on a positional relationship between the plurality of gates and a part of the worker indicated by the estimation result of the pose.

8 . The system according to claim 1 , wherein
the system uses a plurality of the images imaged at mutually-different times to estimate the task at each of the times.

9 . The system according to claim 8 , wherein
the estimation of the task includes estimating the task performed in a prescribed period based on a plurality of the estimation results of the work spot in the prescribed period and a plurality of the estimation results of the state in the prescribed period.

10 . A processing method causing a computer to:
estimate a pose of a worker, a position of an article, an orientation of the article, and a state of the article based on an image of the worker and the article; estimate a work spot of the worker on the article based on an estimation result of the pose, an estimation result of the position, and an estimation result of the orientation; and estimate a task performed by the worker based on an estimation result of the work spot and an estimation result of the state.

11 . A non-transitory computer-readable storage medium storing a program,
the program causing the computer to execute the method according to claim 10 .

Assignees

• Toshiba Kk

Inventors

• Oshima Hirotomo
• Shirakawa Yuta
• Yoshii Takanori
• Kato Takehiro
• Nishimura Keisuke
• Namioka Yasuo

Classifications

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T7/13

  Edge detection · CPC title