Method and device for focusing on target subject, and electronic device

Patent metadata

Field	Value
Publication number	US-2022166930-A1
Application number	US-202217671303-A
Country	US
Kind code	A1
Filing date	Feb 14, 2022
Priority date	Sep 24, 2019
Publication date	May 26, 2022
Grant date	—

Abstract

A method and a device for focusing on a target subject, and an electronic device are provided. The method includes: obtaining a time-of-flight (TOF) image, determining the target subject through subject recognition on the TOF image, obtaining a position information of the target subject in a preview image, and focusing on the target subject according to the position information through a preview lens.

First claim

What is claimed is:

1 . A method for focusing on a target subject, comprising:
obtaining a time-of-flight (TOF) image; determining the target subject through subject recognition on the TOF image; obtaining a position information of the target subject in a preview image; and focusing on the target subject according to the position information through a preview lens.

2 . The method according to claim 1 , wherein the determining the target subject through the subject recognition on the TOF image comprises:
inputting the TOF image into a preset subject detection model to obtain at least two candidate subjects; and determining the target subject from the at least two candidate subjects.

3 . The method according to claim 2 , wherein the determining the target subject from the at least two candidate subjects comprises:
determining a weight of each of the candidate subjects according to a preset weighting rule; and determining one candidate subject with a largest weight as the target subject.

4 . The method according to claim 3 , wherein the preset weighting rule comprises at least one of the following rules:
the weight of the candidate subject increases as a distance between one candidate subject and a TOF lens decreases; the weight of the candidate subject increases as a distance between one candidate subject and an intersection of diagonals of the TOF image decreases; a weight of one human is greater than a weight of one animal, and a weight of one animal is greater than a weight of one plant; and weights of different types of the candidate subjects are determined according to a user instruction.

5 . The method according to claim 2 , wherein the determining the target subject from the at least two candidate subjects comprises:
obtaining a user selection instruction, wherein the user selection instruction is an instruction for a user to select and trigger subject recognition of the at least two candidate subjects; and determining the candidate subject corresponding to the user selection instruction as the target subject.

6 . The method according to claim 2 , wherein the inputting the TOF image into the preset subject detection model to obtain the at least two candidate subjects comprises:
generating a center weight map corresponding to the TOF image, wherein weight values represented by the center weight map gradually decreases from a center to an edge; inputting the TOF image and the center weight map into the preset subject detection model to obtain a subject region confidence map, wherein the preset subject detection model is a model obtained by training according to the TOF image, the center weight map, and a corresponding labeled subject mask pattern for a same scene; and determining the at least two candidate subjects in the TOF image according to the subject region confidence map.

7 . The method according to claim 1 , wherein the obtaining the position information of the target subject in the preview image, and the focusing on the target subject according to the position information through the preview lens comprises:
obtaining a plurality of position coordinates of the target subject in the TOF image; obtaining a plurality of position coordinates of the target subject in the preview image according to a preset correspondence table between a coordinate system of a TOF lens and a coordinate system of the preview lens; and focusing on the target subject according to the position coordinates of the target subject in the preview image through the preview lens.

8 . The method according to claim 1 , wherein the obtaining the position information of the target subject in the preview image, and the focusing on the target subject according to the position information through the preview lens comprises:
obtaining a depth information of the target subject; determining a focus position information of the target subject in the preview image according to the depth information of the target subject; and focusing on the target subject according to the focus position information of the target subject in the preview image through the preview lens.

9 . The method according to claim 1 , further comprising:
obtaining an RGB image through the preview lens; and determining the target subject through the subject recognition on the TOF image and the RGB image.

10 . An electronic device, comprising:
a processor; and a memory configured to store instructions which, when executed by the processor, causes the processor to: obtain a time-of-flight (TOF) image; determine the target subject through subject recognition on the TOF image; obtain a position information of the target subject in a preview image; and focus on the target subject according to the position information through a preview lens.

11 . The electronic device according to claim 10 , wherein the determine the target subject through the subject recognition on the TOF image comprises:
inputting the TOF image into a preset subject detection model to obtain at least two candidate subjects; and determining the target subject from the at least two candidate subjects.

12 . The electronic device according to claim 11 , wherein the determining the target subject from the at least two candidate subjects comprises:
determining a weight of each of the candidate subjects according to a preset weighting rule; and determining one candidate subject with a largest weight as the target subject.

13 . The electronic device according to claim 12 , wherein the preset weighting rule comprises at least one of the following rules:
the weight of the candidate subject increases as a distance between one candidate subject and a TOF lens decreases; the weight of the candidate subject increases as a distance between one candidate subject and an intersection of diagonals of the TOF image decreases; a weight of one human is greater than a weight of one animal, and a weight of one animal is greater than a weight of one plant; and weights of different types of the candidate subjects are determined according to a user instruction.

14 . The electronic device according to claim 11 , wherein the determining the target subject from the at least two candidate subjects comprises:
obtaining a user selection instruction, wherein the user selection instruction is an instruction for a user to select and trigger subject recognition of the at least two candidate subjects; and determining the candidate subject corresponding to the user selection instruction as the target subject.

15 . The electronic device according to claim 11 , wherein the inputting the TOF image into the preset subject detection model to obtain the at least two candidate subjects comprises:
generating a center weight map corresponding to the TOF image, wherein weight values represented by the center weight map gradually decreases from a center to an edge; inputting the TOF image and the center weight map into the preset subject detection model to obtain a subject region confidence map, wherein the preset subject detection model is a model obtained by training according to the TOF image, the center weight map, and a corresponding labeled subject mask pattern for a same scene; and determining the at least two candidate subjects in the TOF image according to the subject region confidence map.

16 . The electronic device according to claim 10 , wherein the obtain the position information of the target subject in the preview image, and the focus on the target subject according to the position information through the preview lens comprises:
ob

Assignees

• Guangdong Oppo Mobile Telecommunications Corp Ltd

Inventors

• Jia Yuhu

Classifications

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• H04N23/675

  comprising setting of focusing regions · CPC title

• H04N23/671

  in combination with active ranging signals, e.g. using light or sound signals emitted toward objects · CPC title

• G06N3/045

  Combinations of networks · CPC title

• H04N23/67

  Focus control based on electronic image sensor signals · CPC title