Information processing apparatus, information processing method, and storage medium

Patent metadata

Field	Value
Publication number	US-2023098633-A1
Application number	US-202217951245-A
Country	US
Kind code	A1
Filing date	Sep 23, 2022
Priority date	Sep 30, 2021
Publication date	Mar 30, 2023
Grant date	—

Abstract

An information processing apparatus performs first determination of determining whether a blurring state of a region in a captured image corresponding to an element of three-dimensional shape data of an object satisfies a condition and performs second determination of, in a case where the blurring state is determined to satisfy the condition, determining whether the region is visible or invisible from an image capture apparatus having captured the captured image and, in a case where the blurring state is determined not to satisfy the condition, determining that the region is invisible. The information processing apparatus generates information used in generating a virtual viewpoint image, based on a determination result by the second determination.

First claim

What is claimed is:

1 . An information processing apparatus comprising:
one or more memories storing instructions: and one or more processors executing the instructions to: perform first determination of determining whether a blurring state of a region in a captured image corresponding to an element forming three-dimensional shape data of an object, generated based on a plurality of captured images captured by a plurality of image capture apparatuses satisfies a condition; perform second determination of, in a case where the blurring state is determined to satisfy the condition, determining whether a part in the object corresponding to the element is captured by an image capture apparatus, and in a case where the blurring state is determined not to satisfy the condition, determining that the part in the object corresponding to the element is not captured by the plurality of image capture apparatuses; and generate information used in generating a virtual viewpoint image, based on a determination result by the second determination.

2 . The information processing apparatus according to claim 1 , wherein the one or more processors further executes the instructions to obtain a blur amount of a part of the object in the captured image and set, as a determination region, a region in which a blurring state is determined based on the blur amount to satisfy the condition, wherein
in a case where a position corresponding to the element is within the determination region, it is determined, by the first determination, that the blurring state of the region in the captured image corresponding to the element satisfies the condition.

3 . The information processing apparatus according to claim 2 , wherein the blur amount is obtained based on a change amount of luminance in the part of the object.

4 . The information processing apparatus according to claim 1 , wherein the one or more processors further executes the instructions to set a three-dimensional region corresponding to a depth of field of the image capture apparatus, in a capture area of the image capture apparatus, wherein
in a case where a position in a three-dimensional space of the element is within a set three-dimensional region, it is determined, by the first determination, that the blurring state of a region in the captured image corresponding to the element satisfies the condition.

5 . The information processing apparatus according to claim 4 , wherein the three-dimensional region is set based on a focus position of the image capture apparatus.

6 . The information processing apparatus according to claim 1 , wherein the one or more processors further executes the instructions to:
obtain a plurality of distance measurement values obtained at a plurality of distance measurement points of the image capture apparatus in focus adjustment and representing a distance from a focus position of the image capture apparatus; and set a three-dimensional region corresponding to a depth of field of the image capture apparatus in a capture area of the image capture apparatus, based on the distance measurement values, wherein in a case where a distance from the image capture apparatus obtained based on a distance measurement value corresponding to the element of the three-dimensional shape data among the plurality of distance measurement values is inside the set three-dimensional region, it is determined, by the first determination, that the blaring state of a region of the captured image corresponding to the element satisfies the condition.

7 . The information processing apparatus according to claim 6 , wherein the three-dimensional region is set, based on a distance measurement value closest to the focus position among the plurality of distance measurement values.

8 . The information processing apparatus according to claim 6 , wherein the one or more processors further executes the instructions to generate a depth image representing a distance to a surface of the object represented by the three-dimensional shape data, wherein
in a case where a difference between a distance measurement value corresponding to the element among the plurality of distance measurement values, and a distance represented by a pixel at a projection position of the element in the depth image is equal to or greater than a threshold value, it is determined, by the second determination, that the part in the object corresponding to the element is not captured.

9 . The information processing apparatus according to claim 1 , wherein the one or more processors further executes the instructions to generate a virtual viewpoint image, based on the plurality of captured images obtained from the plurality of image capture apparatuses, wherein
texture corresponding to the element of the three-dimensional shape data is generated based on a captured image obtained from an image capture apparatus from which the element is determined to be captured by the second determination.

10 . An information processing method comprising:
performing first determination of determining whether a blurring state of a region in a captured image corresponding to an element forming three-dimensional shape data of an object, generated based on a plurality of captured images captured by a plurality of image capture apparatuses satisfies a condition; performing second determination of, in a case where the blurring state is determined to satisfy the condition, determining whether a part in the object corresponding to the element is captured by an image capture apparatus, and in a case where the blurring state is determined not to satisfy the condition, determining that the part in the object corresponding to the element is not captured by the plurality of image capture apparatuses; and generating information used in generating a virtual viewpoint image, based on a determination result by the second determination.

11 . A non-transitory computer-readable storage medium storing a program for causing a computer to execute an information processing method comprising:
performing first determination of determining whether a blurring state of a region in a captured image corresponding to an element forming three-dimensional shape data of an object, generated based on a plurality of captured images captured by a plurality of image capture apparatuses satisfies a condition; performing second determination of, in a case where the blurring state is determined to satisfy the condition, determining whether a part in the object corresponding to the element is captured by an image capture apparatus, and in a case where the blurring state is determined not to satisfy the condition, determining that the part in the object corresponding to the element is not captured by the plurality of image capture apparatuses; and generating information used in generating a virtual viewpoint image, based on a determination result by the second determination.

Assignees

• Canon Kk

Inventors

• Takama Yasufumi

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N13/111 Primary

  Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06T15/205 Primary

  Image-based rendering · CPC title

• H04N23/61

  based on recognised objects · CPC title