Detection device for detecting human-body orientation and detection method for detecting human-body orientation

Patent metadata

Field	Value
Publication number	US-11816860-B2
Application number	US-202117373681-A
Country	US
Kind code	B2
Filing date	Jul 12, 2021
Priority date	Apr 28, 2021
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

A detection device for detecting human-body orientation includes a camera and a processing device. The camera is configured to capture a human-body image. The processing device is configured to cut a human head contour image in the human-body image to obtain an input image, and input the input image to a classifier. The classifier outputs a plurality of human-body orientation probabilities for the input image. The processing device finds the highest human-body orientation probability, and determines whether the highest human-body orientation probability is above the accuracy threshold. In response to the highest human-body orientation probability being above the accuracy threshold, the processing device regards the human-body orientation corresponding to the highest human-body orientation probability as the determined human-body orientation.

First claim

What is claimed is:

1. A detection device for detecting human-body orientation, comprising:
a camera, configured to capture a human-body image; and
a processing device, configured to cut a human head contour image in the human-body image to obtain an input image that does not include the head contour image, and input the input image to a classifier, wherein the classifier outputs a plurality of human-body orientation probabilities for the input image and a plurality of skeleton feature points of the input image;
wherein the processing device finds the highest human-body orientation probability, and determines whether the highest human-body orientation probability is above an accuracy threshold;
in response to the highest human-body orientation probability being above the accuracy threshold, the processing device regards the human-body orientation corresponding to the highest human-body orientation probability as a determined human-body orientation;
in response to the highest human-body orientation probability being below the accuracy threshold, the processing device regards the human-body orientation determined through the skeleton feature points as the determined human-body orientation.

2. The detection device for detecting the human-body orientation of claim 1 , wherein the classifier is implemented by a convolutional neural network (CNN);
after the convolutional neural network receives the input image in the training stage, the convolutional neural network outputs the human-body orientation probabilities and skeleton feature points in the fully connected layer, calculates a regression loss with the human-body orientation corresponding to the highest human-body orientation probability and real orientation data, and calculates a Euclidean distance loss with the skeleton feature points and a plurality of real feature-point position data, then adds the regression loss and the Euclidean distance loss to get a total loss, and adjusts the parameters of the convolutional neural network using a back-propagation method to retrain the convolutional neural network, so that the calculated total loss after each training becomes smaller;
wherein the human-body orientation probabilities correspond to a plurality of feature vectors output by the convolutional neural network.

3. The detection device for detecting the human-body orientation of claim 1 , wherein the human-body orientation probabilities are respectively a frontal body probability, a left-side body probability, a right-side body probability, and a backside body probability.

4. The detection device for detecting the human-body orientation of claim 1 , wherein the skeleton feature points further comprise left shoulder feature point coordinates, right shoulder feature point coordinates, and chest feature point coordinates.

5. The detection device for detecting the human-body orientation of claim 4 , wherein the processing device connects a first straight line between the left shoulder feature point coordinates and the right shoulder feature point coordinates, regards a middle point of the first straight line as a circle center, connects the chest feature point coordinates and the circle center to form a second straight line, calculates an angle of an included angle between the second straight line and the circle center, and select the included angle below an angle threshold to determine the determined human-body orientation;
wherein the angle threshold is an angle of less than 90 degrees and greater than 0 degrees.

6. The detection device for detecting the human-body orientation of claim 5 , wherein in response to the processing device determines that the included angle is equal to 90 degrees, this means that the determined human-body orientation is the front of the human body;
wherein the processing device further determines:
whether the included angle is less than or equal to the angle threshold, in response to the included angle being less than or equal to the angle threshold, and the included angle being located on the left side of the first straight line, the determined human-body orientation is the left side body;
whether the included angle is less than or equal to the angle threshold, in response to the included angle being less than or equal to the angle threshold, and the included angle being located on the right side of the first straight line, the determined human-body orientation is the right side body; and
in response to the processing device determines that the included angles are all greater than the angle threshold, the determined human-body orientation is the front of the human body.

7. The detection device for detecting the human-body orientation of claim 1 , wherein the processing device determines whether there is a human face in the human head contour image in the human-body image;
in response to the processing device determining that there is no face in the head contour image in the human-body image, the determined human-body orientation is the back of the human body.

8. The detection device for detecting the human-body orientation of claim 1 , wherein the processing device counts the determined human-body orientation to obtain usage-habit information.

9. The detection device for detecting the human-body orientation of claim 1 , further comprising:
a display;
wherein the display comprises a display module and an augmented-reality module;
wherein the processing device transmits the determined human-body orientation to the augmented-reality module, the augmented-reality module combines the human-body image with a virtual product according to the determined human-body orientation to generate a combined image, and displays the combined image on the display through the display module.

10. A detection method for detecting human-body orientation, comprising:
capturing a human-body image using a camera;
cutting a human head contour image in the human-body image to obtain an input image that does not include the head contour image, and inputting the input image to a classifier; wherein the classifier outputs a plurality of human-body orientation probabilities for the input image and a plurality of skeleton feature points of the input image;
finding the highest human-body orientation probability, and determining whether the highest human-body orientation probability is above an accuracy threshold; and
in response to the highest human-body orientation probability being above the accuracy threshold, regarding the human-body orientation corresponding to the highest human-body orientation probability as a determined human-body orientation;
in response to the highest human-body orientation probability being below the accuracy threshold, regarding the human-body orientation determined through the skeleton feature points as the determined human-body orientation.

11. The detection method for detecting the human-body orientation of claim 10 , wherein the classifier is implemented by a convolutional neural network (CNN);
after the convolutional neural network receives the input image in the training stage, the convolutional neural network outputs the human-body orientation probabilities and skeleton feature points in the fully connected layer, calculates the regression loss with the human-body orientation corresponding to the highest human-body orientation probability and real orientation data, and calculates the Euclidean distance loss with the skeleton feature points and a plurality of real feature-point position data, then adds the regression loss and the Euclidean distance loss to get the total loss, and adjusts the parameters of the convolutional neural network using a back-propagation method to retrain the convolutional neural

Assignees

• Wistron Corp

Inventors

• Chen Yen Yu

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06V10/267

  by performing operations on regions, e.g. growing, shrinking or watersheds · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title