Method for behaviour recognition, electronic equipment, and storage medium

Patent metadata

Field	Value
Publication number	US-11790692-B2
Application number	US-202117334993-A
Country	US
Kind code	B2
Filing date	May 31, 2021
Priority date	Nov 20, 2020
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

A method and device for behavior recognition are provided. According the present disclosure, an image is collected. It is detected whether the image includes a face. When the image includes a face, it is determined whether the face in the image is a face with closed eyes. When the face in the image is not with closed eyes, a distance between the face and an image collecting unit is computed. A maximum field of vision of a target to which the face belongs is determined according to the distance and a parameter of the image collecting unit. It is determined, according to the maximum field of vision, whether the target has a set behavior.

First claim

What is claimed is:

1. A method for behavior recognition, comprising:
collecting an image, by a camera;
detecting whether the image comprises a face;
in response to the image comprising a face, determining whether the face in the image is a face with closed eyes, wherein the face in the image is a face of a target;
in response to determining that the face in the image is not with closed eyes, computing a distance between the face and the camera, comprising:
determining a size of a face image; and
computing the distance between the face of the target and the camera based on the size of the face image and a resolution of a display of the camera;

determining a maximum field of vision of the target according to the distance and a parameter of the camera, comprising:
computing a field of view of the camera according to the parameter of the camera, wherein the parameter of the camera comprises a focal length of the camera, a physical height of a sensor of the camera and a physical width of the sensor of the camera;
acquiring a location of eyes of the face;
determining a coordinate of a midpoint of a line connecting pupils of a left eye and a right eye; and
respectively computing a maximum field of vision of the target in a horizontal direction and a maximum field of vision of the target in a vertical direction based on the coordinate, the size of the face, the field of view, a size of a target to be operated, and the distance; and

determining whether the target has a set behavior according to the maximum field of vision.

2. The method of claim 1 , further comprising:
expanding the maximum field of vision in the horizontal direction by a first angular range; and
expanding the maximum field of vision in the vertical direction by a second angular range.

3. The method of claim 1 , further comprising:
acquiring a yaw and a pitch of the face; and
wherein determining whether the target has the set behavior according to the maximum field of vision comprises:
determining, according to the yaw, the pitch, the maximum field of vision in the horizontal direction, and the maximum field of vision in the vertical direction, whether the target has the set behavior,
wherein in response to the yaw being located in the maximum field of vision in the horizontal direction and the pitch being located in the maximum field of vision in the vertical direction, the target is determined to have the set behavior.

4. The method of claim 1 , further comprising:
in response to the image comprising multiple faces of multiple targets, determining that the target has the set behavior in response to at least one of the multiple targets having the set behavior.

5. The method of claim 1 , further comprising:
generating an operating instruction in response to the target having the set behavior, the operating instruction being configured to adjust a target to be operated to a first state.

6. The method of claim 5 , wherein the first state comprises at least one of:
an awake state, or a power-on state.

7. An electronic equipment comprising: a processor and a memory for storing processor executable instructions, wherein the processor is configured, by calling the executable instructions in the memory, to implement:
collecting an image, by a camera of the electronic equipment;
detecting whether the image comprises a face;
in response to the image comprising a face, determining whether the face in the image is a face with closed eyes, wherein the face in the image is a face of a target;
in response to determining that the face in the image is not with closed eyes, computing a distance between the face and the camera;
determining a maximum field of vision of a target to which the face belongs according to the distance and a parameter of the camera; and
determining whether the target has a set behavior according to the maximum field of vision,
wherein the processor is configured to compute the distance between the face and the camera by:
determining a size of a face image; and
computing the distance between the face of the target and the camera based on the size of the face image and a resolution of a display of the camera; and
wherein the processor is configured to determine the maximum field of vision of the target to which the face belongs according to the distance and the parameter of the camera by:
computing a field of view of the camera according to the parameter of the camera, wherein the parameter of the camera comprises a focal length of the camera, a physical height of a sensor of the camera and a physical width of the sensor of the camera;
acquiring a location of eyes of the face;
determining a coordinate of a midpoint of a line connecting pupils of a left eye and a right eye; and
respectively computing a maximum field of vision of the target in a horizontal direction and a maximum field of vision of the target in a vertical direction based on the coordinate, the size of the face, the field of view, a size of a target to be operated, and the distance.

8. The electronic equipment of claim 7 , wherein the processor is further configured to implement:
expanding the maximum field of vision in the horizontal direction by a first angular range; and
expanding the maximum field of vision in the vertical direction by a second angular range.

9. The electronic equipment of claim 7 , wherein the processor is further configured to implement:
acquiring a yaw and a pitch of the face; and
wherein the processor is configured to determine whether the target has the set behavior according to the maximum field of vision by:
determining, according to the yaw, the pitch, the maximum field of vision in the horizontal direction, and the maximum field of vision in the vertical direction, whether the target has the set behavior,
wherein in response to the yaw being located in the maximum field of vision in the horizontal direction and the pitch being located in the maximum field of vision in the vertical direction, the target is determined to have the set behavior.

10. The electronic equipment of claim 7 , wherein the processor is further configured to implement:
in response to the image comprising multiple faces of multiple targets, determining that the target has the set behavior in response to at least one of the multiple targets having the set behavior.

11. The electronic equipment of claim 7 , wherein the processor is further configured to implement:
generating an operating instruction in response to the target having the set behavior,
the operating instruction being configured to adjust a target to be operated to a first state.

12. The electronic equipment of claim 11 , wherein the first state comprises at least one of:
an awake state, or a power-on state.

13. A non-transitory computer-readable storage medium having stored therein instructions which, when executed by a processor of electronic equipment, allow the electronic equipment to implement:
collecting an image, by a camera;
detecting whether the image comprises a face;
in response to the image comprising a face, determining whether the face in the image is a face with closed eyes, wherein the face in the image is a face of a target;
in response to determining that the face in the image is not with closed eyes, computing a distance between the face and the camera;
determining a maximum field of vision of the target according to the distance and a parameter of the camera; and
determining whether the target has a set behavior according to the maximum field of vision,
wherein computing the distance between the face and the camera comprises:
determining a size of a face image; an

Assignees

• Xiaomi Technology Wuhan Co Ltd
• Beijing Xiaomi Pinecone Electronics Co Ltd

Inventors

• Hu Xian
• Deng Wei
• Yi Jun

Classifications

• G06V40/161 Primary

  Detection; Localisation; Normalisation · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/17

  using hand-held instruments · CPC title

• G06V40/165

  using facial parts and geometric relationships · CPC title

• G06V40/171 Primary

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title