Generating an augmented reality image using a blending factor

Patent metadata

Field	Value
Publication number	US-12444145-B2
Application number	US-202318519416-A
Country	US
Kind code	B2
Filing date	Nov 27, 2023
Priority date	Jun 17, 2016
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

A method for generating an augmented reality image from first and second images, wherein at least a portion of at least one of the first and the second image is captured from a real scene, identifies a confidence region in which a confident determination as to which of the first and second image to render in that region of the augmented reality image can be made, and identifies an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the augmented reality image. At least one blending factor value in the uncertainty region is determined based upon a similarity between a first colour value in the uncertainty region and a second colour value in the confidence region, and an augmented reality image is generated by combining, in the uncertainty region, the first and second images using the at least one blending factor value.

First claim

What is claimed is:

1. A method for generating a composite image from first and second images, the composite image, the first image and the second image each comprising a first region and a second region, the method comprising:
determining at least one blending factor value, for use in generating the first region of the composite image, based upon, in one of the first and second images, a similarity between a first value in the first region and a second value in the second region; and
generating the composite image by combining, in the first region of the composite image, the first and second images using the at least one blending factor value;
wherein the first region of the composite image is an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the composite image.

2. The method of claim 1 , wherein at least a portion of at least one of the first image and second image is captured from a real scene.

3. The method of claim 1 , wherein the first region of the composite image corresponds to the respective first regions of each of the first and second image.

4. The method of claim 1 , wherein a determination as to which of the first and second images to render can be made for a second region of the composite image, wherein the second region of the composite image corresponds to the respective second regions of each of the first and second image.

5. The method of claim 1 , wherein the second region of the composite image is a confidence region in which a confident determination as to which of the first and second image to render in that region of the composite image can be made.

6. The method of claim 1 , wherein the first and second values are colour values.

7. The method of claim 1 , further comprising:
identifying the first region of the composite image; and
identifying the second region of the composite image.

8. The method of claim 1 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, wherein the method further comprises making said determination as to which of the first image and the second image to render based upon a depth value of the first image and the corresponding depth value of the second image in the second region.

9. The method of claim 7 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, and wherein the first region is identified based upon at least one depth value associated with at least one of the first and the second image, the at least one depth value being derived from a depth value captured from the real scene.

10. The method of claim 1 , further comprising generating at least one initial blending factor value in a second region based upon said determination and wherein generating the composite image further comprises combining a corresponding colour value of the first image and a corresponding colour value of the second image in the second region using the at least one initial blending factor value.

11. The method of claim 5 , further comprising identifying the confidence region, wherein identifying the confidence region comprises categorising portions of the confidence region as first confidence regions or second confidence regions, wherein:
first confidence regions are confidence regions in which a colour value of the first image is to be rendered in the corresponding region of the composite image; and
second confidence regions are confidence regions in which a colour value of the second image is to be rendered in the corresponding region of the composite image.

12. The method of claim 1 , wherein determining the at least one blending factor value is further based upon the distance between the position of the first value and the position of the second value.

13. The method of claim 1 , wherein the first region, of the at least one of the first image and second image that is captured from a real scene, comprises a plurality of sample points and determining the at least one blending factor value further comprises processing, for each of a plurality of sample points in the said first region, that sample point based upon values at a plurality of sample points located in a second region, of the said at least one of the first image and second image, within a predetermined distance of that sample point.

14. The method of claim 1 , wherein determining at least one blending factor value for the first region of the composite image is based upon a similarity between a colour value in the said first region and at least one corresponding colour value of each of the first image and the second image.

15. The method of claim 1 , further comprising performing an erosion operation on the second region of the composite image, wherein the erosion operation is configured to re-categorise at least one portion of the second region of the composite image as forming a part of a first region of the composite image.

16. The method of claim 1 , wherein the first image is a captured image of a real scene and the second image is an image of a virtual object.

17. An image processing system for generating a composite image from first and second images, the composite image, the first and the second image each comprising a first region and a second region, the image processing system comprising:
a blend module arranged to determine at least one blending factor value, for use in generating the first region of the composite image, based upon, in one of the first and second images, a similarity between a first value in the first region and a second value in the second region; and
an image generation module arranged to generate the composite image by combining, in the first region of the composite image, the first and second images using the at least one blending factor value;
wherein the first region of the composite image is an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the composite image.

18. The image processing system of claim 17 wherein the second region of the composite image is a confidence region in which a confident determination as to which of the first and second image to render in that region of the composite image can be made, and wherein the image processing system further comprises:
an uncertainty identification module arranged to identify the uncertainty region; and
a confidence identification module arranged to identify the confidence region.

19. A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to generate a composite image from first and second images, the composite image, the first image and the second image each comprise a first region and a second region, the instructions causing the computer system to generate the composite image by:
determining at least one blending factor value, for use in generating the first region of the composite image, based upon, in one of the first and second images, a similarity between a first value in the first region and a second value in the second region; and
generating the composite image by combining, in the first region of the composite image, the first and second images using the at least one blending factor value;
wherein the first region of the composite image is an uncertainty region in which it is uncertain as to which of the fir

Assignees

• Imagination Tech Ltd

Inventors

• Walton David

Classifications

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T7/30

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06T15/503

  Blending, e.g. for anti-aliasing · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title