Image processing apparatus and method, and image capturing apparatus

Patent metadata

Field	Value
Publication number	US-12401898-B2
Application number	US-202318335180-A
Country	US
Kind code	B2
Filing date	Jun 15, 2023
Priority date	Jun 27, 2022
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

An image processing apparatus comprises: a first detector that detects a subject area of a predetermined subject from each of a plurality of images; a second detector that detects a partial area of a predetermined size that includes the subject area from each of the plurality of images; a feature point detector that detects a feature point of an image; and a synthesizer that synthesizes the partial areas of the plurality of images so that the feature points in the subject areas match each other.

First claim

What is claimed is:

1. An image processing apparatus comprising one or more processors and/or circuitry which function as:
a first detector that detects a subject area of a predetermined subject from each of a plurality of images;
a second detector that detects a partial area of a predetermined size that includes the subject area from each of the plurality of images;
a feature point detector that detects a feature point of an image;
a determination unit that determines facial deformation amount in a case where the subject area is of a person's face; and
a synthesizer that synthesizes the partial areas of the plurality of images so that the feature points in the subject areas match each other,
wherein, with respect to the subject area of the partial area, the synthesizer selects the subject areas including a person's face whose facial deformation amount is smaller than a predetermined threshold from the subject areas of the plurality of images and synthesizes the selected subject areas.

2. The image processing apparatus according to claim 1 , wherein the second detector detects an area where a moving amount of the feature point, detected around the subject area, between consecutive images is larger than a predetermined moving amount as the partial area.

3. The image processing apparatus according to claim 1 , wherein the one or more processors and/or circuitry further function as a second determination unit that determines facial expression in a case where the subject area is of a person's face,
wherein, with respect to the subject area of the partial area, the synthesizer selects the subject areas including a person's face whose facial expression satisfies a predetermined condition from the subject areas of the plurality of images and synthesizes the selected subject areas.

4. The image processing apparatus according to claim 1 , wherein the synthesizer further synthesizes the synthesized partial area with an area other than the partial area in one of the plurality of images.

5. The image processing apparatus according to claim 1 , wherein the one or more processors and/or circuitry further function as a third detector that detects a moving amount of the partial area between consecutive images among the plurality of images, and
the synthesizer selects the partial areas whose moving amount is larger than a predetermined moving amount from the partial areas of the plurality of images and synthesizes the selected partial areas.

6. The image processing apparatus according to claim 1 , wherein the one or more processors and/or circuitry further function as:
a third detector that detects a moving amount of the subject area between consecutive images among the plurality of images; and
a setting unit that sets a size of the partial area based on the moving amount,
wherein the setting unit sets the size of the partial area to be smaller in a case where the moving amount is a second moving amount which is larger than a first moving amount than in a case where the moving amount is the first moving amount.

7. The image processing apparatus according to claim 6 , wherein the one or more processors and/or circuitry further function as a determination unit that determines a deformation amount between the subject area of a reference image to be referenced among the plurality of images and the subject area of an image other than the reference images,
wherein the synthesizer selects the partial area including the subject area whose deformation amount is smaller than a predetermined threshold and synthesizes the selected partial area with the partial area of the reference image.

8. An image capturing apparatus comprising:
an image sensor; and
one or more processors and/or circuitry which function as:
a first detector that detects a subject area of a predetermined subject from each of a plurality of images shot by the image sensor;
a second detector that detects a partial area of a predetermined size that includes the subject area from each of the plurality of images;
a feature point detector that detects a feature point of an image;
a determination unit that determines facial deformation amount in a case where the subject area is of a person's face; and
a synthesizer that synthesizes the partial areas of the plurality of images so that the feature points in the subject areas match each other,

wherein, with respect to the subject area of the partial area, the synthesizer selects the subject areas including a person's face whose facial deformation amount is smaller than a predetermined threshold from the subject areas of the plurality of images and synthesizes the selected subject areas.

9. An image processing method comprising:
detecting a subject area of a predetermined subject from each of a plurality of images;
detecting a partial area of a predetermined size that includes the subject area from each of the plurality of images;
detecting a feature point of an image;
determining facial deformation amount in a case where the subject area is of a person's face; and
synthesizing the partial areas of the plurality of images so that the feature points in the subject areas match each other,
wherein, with respect to the subject area of the partial area, in the synthesizing, the subject areas including a person's face whose facial deformation amount is smaller than a predetermined threshold are selected from the subject areas of the plurality of images and the selected subject areas are synthesized.

10. An image processing apparatus comprising one or more processors and/or circuitry which function as:
a first detector that detects a subject area of a predetermined subject from each of a plurality of images;
a second detector that detects a partial area of a predetermined size that includes the subject area from each of the plurality of images;
a feature point detector that detects a feature point of an image;
a third detector that detects a moving amount of the subject area between consecutive images among the plurality of images;
a setting unit that sets a size of the partial area based on the moving amount; and
a synthesizer that synthesizes the partial areas of the plurality of images so that the feature points in the subject areas match each other,
wherein the setting unit sets the size of the partial area to be smaller in a case where the moving amount is a second moving amount which is larger than a first moving amount than in a case where the moving amount is the first moving amount.

11. The image processing apparatus according to claim 10 , wherein the one or more processors and/or circuitry further function as a determination unit that determines a deformation amount between the subject area of a reference image to be referenced among the plurality of images and the subject area of an image other than the reference images,
wherein the synthesizer selects the partial area including the subject area whose deformation amount is smaller than a predetermined threshold and synthesizes the selected partial area with the partial area of the reference image.

12. An image capturing apparatus comprising:
an image sensor; and
one or more processors and/or circuitry which function as:
a first detector that detects a subject area of a predetermined subject from each of a plurality of images shot by the image sensor;
a second detector that detects a partial area of a predetermined size that includes the subject area from each of the plurality of images;
a feature point detector that detects a feature point of an image;
a synthesizer that synthesizes the partial areas of the plurality of images so that the feature points in the subject areas match each other;

Assignees

• Canon Kk

Inventors

• Saito Junichi

Classifications

• G06T7/215

  Motion-based segmentation · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• H04N5/265

  Mixing · CPC title

• H04N23/6811 Primary

  based on the image signal · CPC title

• H04N23/611 Primary

  where the recognised objects include parts of the human body · CPC title