Processing system, estimation apparatus, processing method, and non-transitory storage medium

Patent metadata

Field	Value
Publication number	US-12039772-B2
Application number	US-201917600711-A
Country	US
Kind code	B2
Filing date	Apr 5, 2019
Priority date	Apr 5, 2019
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

The present invention provides a processing system (10) including: a sample image generation unit (11) that generates a plurality of sample images being each associated with a partial region of a first image generated using a first lens; an estimation unit (12) that generates an image content estimation result indicating a content for each of the sample images using an estimation model generated by machine learning using a second image generated using a second lens differing from the first lens; a task execution unit (14) that estimates a relative positional relationship of a plurality of the sample images in the first image; a determination unit (15) that determines whether an estimation result of the relative positional relationship is correct; and a correction unit (16) that corrects a value of a parameter of the estimation model when the estimation result of the relative positional relationship is determined to be incorrect.

First claim

What is claimed is:

1. A processing system comprising:
at least one memory configured to store one or more instructions; and
at least one processor configured to execute the one or more instructions to:
generate, from a first image for learning generated by capture using a first lens, a plurality of sample images being each associated with a partial region of the first image for learning;
input the sample image into an estimation model generated by machine learning using learning data including a second image generated by capture using a second lens differing in characteristic from the first lens and a label indicating a content of the second image, and generate an image content estimation result indicating a content for each of the sample images;
estimate, based on the image content estimation result for each of the sample images, a relative positional relationship of a plurality of the sample images in the first image for learning;
determine whether an estimation result of the relative positional relationship is correct; and
correct a value of a parameter of the estimation model when an estimation result of the relative positional relationship is determined to be incorrect.

2. The processing system according to claim 1 , wherein
the processor is further configured to execute the one or more instructions to correct a value of a parameter of the estimation model, based on a stochastic gradient descent method.

3. The processing system according to claim 1 ,
wherein the processor is further configured to execute the one or more instructions to iteratively execute the generating a plurality of sample images; the inputting the sample image into the estimation model, the generating the image content estimation result, the estimating the relative positional relationship of a plurality of the sample images, the determining whether the estimation result of the relative positional relationship is correct, and correcting the value of the parameter of the estimation model, until the estimation result of the relative positional relationship satisfies an end condition.

4. The processing system according to claim 1 , wherein
the first lens is a fish-eye lens, and the second lens is a lens differing from a fish-eye lens.

5. The processing system according to claim 4 , wherein
the processor is further configured to execute the one or more instructions to extract, as the sample image, a partial region in a panoramic image for learning resulting from plane development of the first image for learning generated by capture using a fish-eye lens.

6. The processing system according to claim 5 ,
wherein the processor is further configured to execute the one or more instructions to apply, by transfer learning using learning data including a fish-eye lens image for transfer learning generated by capture using a fish-eye lens and a label indicating a content of the fish-eye lens image for transfer learning, the estimation model for estimating a content of the panoramic image, to a region for estimating a content of the fish-eye lens image.

7. A processing method executed by a computer, the method comprising:
generating, from a first image for learning generated by capture using a first lens, a plurality of sample images being each associated with a partial region of the first image for learning;
inputting the sample image into an estimation model generated by machine learning using learning data including a second image generated by capture using a second lens differing in characteristic from the first lens and a label indicating a content of the second image, and generating an image content estimation result indicating a content for each of the sample images;
estimating, based on the image content estimation result for each of the sample images, a relative positional relationship of a plurality of the sample images in the first image for learning;
determining whether an estimation result of the relative positional relationship is correct; and
correcting a value of a parameter of the estimation model when an estimation result of the relative positional relationship is determined to be incorrect.

8. A non-transitory storage medium storing a program that causes a computer to:
generate, from a first image for learning generated by capture using a first lens, a plurality of sample images being each associated with a partial region of the first image for learning;
input the sample image into an estimation model generated by machine learning using learning data including a second image generated by capture using a second lens differing in characteristic from the first lens and a label indicating a content of the second image, and generate an image content estimation result indicating a content for each of the sample images;
estimate, based on the image content estimation result for each of the sample images, a relative positional relationship of a plurality of the sample images in the first image for learning;
determine whether an estimation result of the relative positional relationship is correct; and
correct a value of a parameter of the estimation model when an estimation result of the relative positional relationship is determined to be incorrect.

Assignees

• Nec Corp

Inventors

• Liu Jianquan
• Li Junnan

Classifications

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T5/60

  using machine learning, e.g. neural networks · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/243

  by compensating for image skew or non-uniform image deformations · CPC title