Creating a realistic color for a virtual object in an augmented reality environment

Patent metadata

Field	Value
Publication number	US-8976191-B1
Application number	US-201414210053-A
Country	US
Kind code	B1
Filing date	Mar 13, 2014
Priority date	Mar 13, 2014
Publication date	Mar 10, 2015
Grant date	Mar 10, 2015

Abstract

Disclosed is a method and apparatus for creating a realistic color for a virtual object in an Augmented Reality environment produced by an Augmented Reality application. In one embodiment, the functions implemented include: selecting a reference image target frame; selecting a plurality of sample points in the reference image target frame; acquiring a subsequent new image target frame; determining a plurality of corresponding sample points in the new image target frame wherein the plurality of corresponding sample points correspond to the plurality of sample points in the reference image target frame; comparing a color of each of the plurality of sample points in the reference image target frame with a color of each of the corresponding sample points in the new image target frame and computing a Color Transfer function based at least in part on the comparison; and applying the Color Transfer function to the color of the virtual object.

First claim

What is claimed is:

1. A method of creating a realistic color for a virtual object in an Augmented Reality environment comprising:
selecting a reference image target frame;
selecting a plurality of sample points in the reference image target frame;
acquiring a subsequent new image target frame;
determining a plurality of corresponding sample points in the new image target frame wherein the plurality of corresponding sample points correspond to the plurality of sample points in the reference image target frame;
comparing a color of each of the plurality of sample points in the reference image target frame with a color of each of the corresponding sample points in the new image target frame and computing a Color Transfer function based at least in part on the comparison; and
applying the Color Transfer function to the color of the virtual object,
wherein the Color Transfer function is based upon a color measurement of a sample point in the reference image target frame, a color measurement of a sample point in the new image target frame, and a color of the virtual object before Color Transfer.

2. The method of claim 1 , wherein the plurality of sample points selected are the same as at least some of the keypoints of the reference image target frame.

3. The method of claim 1 , wherein the plurality of sample points are selected to maximize a range of colors covered by the plurality of sample points.

4. The method of claim 1 , wherein the plurality of sample points are selected to span a substantially large portion of the reference image target frame.

5. The method of claim 1 , wherein the Color Transfer function is of a form of F(R, S, c)→t, wherein R is a color measurement in the reference image target frame, wherein S is a color measurement in the new image target frame, wherein c is a color of the virtual object before Color Transfer, and wherein t is a resulting color of c after Color Transfer.

6. The method of claim 5 , wherein the Color Transfer function is further of a form of ti=(Si−Ri)+ci, wherein ci is the i-th color of the virtual object, wherein Ri is a color measurement of a sample point whose color most closely resembles ci, wherein Si is a color measurement of a corresponding sample point corresponding to Ri, and wherein ti is a resulting color of ci after Color Transfer.

7. The method of claim 5 , wherein the Color Transfer function is further of a form of

t
=

1
n

⁢

∑

j
=
1

n

⁢

⁢

(

S
j

-

R
j

)

+
c

,

wherein n is the number of sample points in the reference image target frame.

8. The method of 1 , wherein applying the Color Transfer function further comprises applying the Color Transfer function in an intermediary color space, and converting a resulting color to a target color space.

9. An apparatus for creating a realistic color for a virtual object in an Augmented Reality environment comprising:
a memory; and
a processor, coupled to the memory, configured to execute instructions to:
select a reference image target frame;
select a plurality of sample points in the reference image target frame;
acquire a subsequent new image target frame;
determine a plurality of corresponding sample points in the new image target frame wherein the plurality of corresponding sample points correspond to the plurality of sample points in the reference image target frame;
compare a color of each of the plurality of sample points in the reference image target frame with a color of each of the corresponding sample points in the new image target frame and compute a Color Transfer function based at least on part on the comparison; and
apply the Color Transfer function to the color of the virtual object,
wherein the Color Transfer function is based upon a color measurement of a sample point in the reference image target frame, a color measurement of a sample point in the new image target frame, and a color of the virtual object before Color Transfer.

10. The apparatus of claim 9 , wherein the plurality of sample points selected are the same as at least some of the keypoints of the reference image target frame.

11. The apparatus of claim 9 , wherein the plurality of sample points are selected to maximize a range of colors covered by the plurality of sample points.

12. The apparatus of claim 9 , wherein the plurality of sample points are selected to span a substantially large portion of the reference image target frame.

13. The apparatus of claim 9 , wherein the Color Transfer function is of a form of F(R, S, c)→t, wherein R is a color measurement in the reference image target frame, wherein S is a color measurement in the new image target frame, wherein c is a color of the virtual object before Color Transfer, and wherein t is a resulting color of c after Color Transfer.

14. The apparatus of claim 13 , wherein the Color Transfer function is further of a form of ti=(Si−Ri)+ci, wherein ci is the i-th color of the virtual object, wherein Ri is a color measurement of a sample point whose color most closely resembles ci, wherein Si is a color measurement of a corresponding sample point corresponding to Ri, and wherein ti is a resulting color of ci after Color Transfer.

15. The apparatus of claim 13 , wherein the Color Transfer function is further of a form of

t
=

1
n

⁢

∑

j
=
1

n

⁢

⁢

(

S
j

-

R
j

)

+
c

Assignees

• Qualcomm Inc
• Qualcomm Inc

Inventors

• Mendez Mendez Erick

Classifications

• G09G5/026 Primary

  Control of mixing and/or overlay of colours in general (G09G5/022 and G09G5/024 take precedence) · CPC title

• G09G2340/06

  Colour space transformation · CPC title

• G09G2340/12

  Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels · CPC title

• G06T2215/16

  Using real world measurements to influence rendering · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title