Generating an augmented reality image using a blending factor

Patent metadata

Field	Value
Publication number	US-11087554-B2
Application number	US-202016794041-A
Country	US
Kind code	B2
Filing date	Feb 18, 2020
Priority date	Jun 17, 2016
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

A method for generating an augmented reality image from first and second images, wherein at least a portion of at least one of the first and the second image is captured from a real scene, identifies a confidence region in which a confident determination as to which of the first and second image to render in that region of the augmented reality image can be made, and identifies an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the augmented reality image. At least one blending factor value in the uncertainty region is determined based upon a similarity between a first colour value in the uncertainty region and a second colour value in the confidence region, and an augmented reality image is generated by combining, in the uncertainty region, the first and second images using the at least one blending factor value.

First claim

What is claimed is:

1. A method for generating an augmented reality image from first and second images, wherein at least a portion of at least one of the first and the second image is captured from a real scene, and wherein at least one of the first and the second image comprises an image of a virtual object, the method comprising:
determining at least one blending factor value in a first region based upon a similarity between a first value in the first region and at least one second value in a second region, wherein in the second region a determination as to which of the first and second image to render in that region of the augmented reality image can be made; and
generating an augmented reality image by combining, in the first region of the augmented reality image, the first and second images using the at least one blending factor value.

2. The method of claim 1 wherein the first region is an uncertainty region of the augmented reality image, in which it is uncertain as to which of the first and second image to render in that region of the augmented reality image.

3. The method of claim 1 wherein the second region is a confidence region in which a confident determination as to which of the first and second image to render in that region of the augmented reality image can be made.

4. The method of claim 1 wherein the first and second values are colour values.

5. The method of claim 1 further comprising:
identifying the first region; and
identifying the second region.

6. The method according to claim 1 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, wherein the method further comprises making said determination as to which of the first image and the second image to render based upon a depth value of the first image and the corresponding depth value of the second image in the second region.

7. The method according to claim 5 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, and wherein the first region is identified based upon at least one depth value associated with at least one of the first and the second image, the at least one depth value being derived from a depth value captured from a real scene.

8. The method according to claim 1 , further comprising generating at least one initial blending factor value in a second region based upon said determination and wherein generating the augmented reality image further comprises combining a corresponding colour value of the first image and a corresponding colour value of the second image in the second region using the at least one initial blending factor value.

9. The method according to claim 3 further comprising identifying the confidence region, wherein identifying the confidence region comprises categorising portions of the confidence region as first confidence regions or second confidence regions, wherein:
first confidence regions are confidence regions in which a colour value of the first image is to be rendered in the corresponding region of the augmented reality image; and
second confidence regions are confidence regions in which a colour value of the second image is to be rendered in the corresponding region of the augmented reality image.

10. The method according to claim 1 , wherein determining the at least one blending factor value is further based upon the distance between the position of the first value and the position of the at least one second value.

11. The method according to claim 1 , wherein the first region comprises a plurality of sample points and determining the at least one blending factor value further comprises processing, for each of a plurality of sample points in the first region, that sample point based upon values at a plurality of sample points located in a second region within a predetermined distance of that sample point.

12. The method according to claim 1 , wherein determining at least one blending factor value in the first region is based upon a similarity between a colour value in the first region and at least one corresponding colour value of each of the first image and the second image.

13. The method according to claim 1 , further comprising performing an erosion operation on the second region, wherein the erosion operation is configured to re-categorise at least one portion of the second region as forming a part of a first region.

14. The method according to claim 1 , wherein the first image is a captured image of a real scene and the second image is an image of a virtual object.

15. An augmented reality processing system for generating an augmented reality image from first and second images, wherein at least a portion of at least one of the first and the second image is captured from a real scene, and wherein at least one of the first and the second image comprises an image of a virtual object, the augmented reality processing system comprising:
a blend module arranged to determine at least one blending factor value in a first region based upon a similarity between a first value in the first region and at least one second value in a second region, wherein in the second region a determination as to which of the first and second image to render in that region of the augmented reality image can be made; and
an image generation module arranged to generate an augmented reality image by combining, in the first region of the augmented reality image, the first and second images using the at least one blending factor value.

16. The augmented reality processing system of claim 15 wherein the second region is a confidence region in which a confident determination as to which of the first and second image to render in that region of the augmented reality image can be made, and wherein the augmented reality processing system further comprises a confidence identification module arranged to identify the confidence region.

17. The augmented reality processing system of claim 15 wherein the first region is an uncertainty region in which it is uncertain as to which of the first and second image to render in that region of the augmented reality image, and wherein the augmented reality processing system further comprises an uncertainty identification module arranged to identify the uncertainty region.

18. An augmented reality processing system according to claim 15 , wherein the first image and the second image each have associated therewith a plurality of colour values and a corresponding plurality of depth values, and wherein the first and second values are colour values.

19. The augmented reality processing system according to claim 13 , wherein the augmented reality processing system is embodied in hardware on an integrated circuit.

20. A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to generate an augmented reality image from first and second images by:
determining at least one blending factor value in a first region based upon a similarity between a first value in the first region and at least one second value in a second region, wherein in the second region a determination as to which of the first and second image to render in that region of the augmented reality image can be made; and
generating an augmented reality image by combining, in the first region of the augmented reality image, the first and second im

Assignees

• Imagination Tech Ltd

Inventors

• Walton David

Classifications

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06T15/503

  Blending, e.g. for anti-aliasing · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title