Augmented reality image generation

Patent metadata

Field	Value
Publication number	US-11501498-B2
Application number	US-201815974604-A
Country	US
Kind code	B2
Filing date	May 8, 2018
Priority date	May 8, 2018
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

According to one implementation, an augmented reality image generation system includes a display, and a computing platform having a hardware processor and a system memory storing a software code. The hardware processor executes the software code to receive a camera image depicting one or more real-world object(s), and to identify one or more reference point(s) corresponding to the camera image, each of the reference point(s) having a predetermined real-world location. The software code further maps the real-world object(s) to their respective real-world location(s) based on the predetermined real-world location(s) of the reference point(s), merges the camera image with a virtual object to generate an augmented reality image including the real-world object(s) and the virtual object, and renders the augmented reality image on the display. The location of the virtual object in the augmented reality image is determined based on the real-world location(s) of the real-world object(s).

First claim

What is claimed is:

1. An augmented reality image generation system comprising:
a computing platform including a hardware processor and a system memory;
a software code stored in the system memory;
a display communicatively coupled to the computing platform;
the hardware processor configured to execute the software code to:
obtain a reference point metadata associated with a first camera image, the reference point metadata including a timestamp associated with the first camera image;
receive a second camera image depicting at least one real-world object;
identify at least one reference point depicted in the second camera image based on the reference point metadata including the timestamp associated with the first camera image, the at least one reference point having a predetermined real-world location, the at least one reference point is separate from the at least one real-world object;
map the at least one real-world object to a real-world location of the at least one real-world object based on the predetermined real-world location of the at least one reference point;
merge the second camera image with a virtual object to generate an augmented reality image including the at least one real-world object and the virtual object, wherein a location of the virtual object in the augmented reality image is determined based on the real-world location of the at least one real-world object; and
render the augmented reality image on the display.

2. The augmented reality image generation system of claim 1 , wherein the augmented reality image is rendered as a three-hundred-and-sixty degree (360°) image.

3. The augmented reality image generation system of claim 1 , wherein identifying the at least one reference point depicted in the second camera image further comprises performing an image analysis of the second camera image.

4. The augmented reality image generation system of claim 3 , wherein the hardware processor is further configured to execute the software code to:
receive a sensor data corresponding to the at least one real-world object; and
map the at least one real-world object to the real-world location of the at least one real-world object based on the predetermined real-world location of the at least one reference point and the sensor data.

5. The augmented reality image generation system of claim 1 , wherein the second camera image is captured indoors.

6. The augmented reality image generation system of claim 1 , wherein the hardware processor is further configured to execute the software code to:
receive a sensor data corresponding to the at least one real-world object; and
map the at least one real-world object to the real-world location of the at least one real-world object based on the predetermined real-world location of the at least one reference point and the sensor data.

7. The augmented reality image generation system of claim 6 , wherein the sensor data comprises at least one of depth sensor data, accelerometer data, or magnetometer data.

8. A method for use by an augmented reality image generation system including a computing platform having a hardware processor, a system memory storing a software code, and a display, the method comprising:
obtaining, using the hardware processor, a reference point metadata associated with a first camera image, the reference point metadata including a timestamp associated with the first camera image;
receiving, using the hardware processor, a second camera image depicting at least one real-world object;
identifying, using the hardware processor, at least one reference point depicted in the second camera image based on the reference point metadata including the timestamp associated with the first camera image, the at least one reference point having a predetermined real-world location, the at least one reference point is separate from the at least one real-world object;
mapping, using the hardware processor, the at least one real-world object to a real-world location of the at least one real-world object based on the predetermined real-world location of the at least one reference point;
merging, using the hardware processor, the second camera image with a virtual object to generate an augmented reality image including the at least one real-world object and the virtual object, wherein a location of the virtual object in the augmented reality image is determined based on the real-world location of the at least one real-world object; and
rendering, using the hardware processor, the augmented reality image on the display.

9. The method of claim 8 , wherein the augmented reality image is rendered as a three-hundred-and-sixty degree (360°) image.

10. The method of claim 8 , wherein identifying the at least one reference point depicted in the second camera image further comprises performing an image analysis of the second camera image.

11. The method of claim 8 , wherein the second camera image is captured indoors.

12. The method of claim 8 , further comprising:
receiving, using the hardware processor, a sensor data corresponding to the at least one real-world object; and
mapping, using the hardware processor, the at least one real-world object to the real-world location of the at least one real-world object based on the predetermined real-world location of the at least one reference point and the sensor data.

13. The method of claim 12 , wherein the sensor data comprises at least one of depth sensor data, accelerometer data, or magnetometer data.

14. A mobile user system comprising:
a hardware processor, a system memory, a display, and a camera;
a software code stored in the system memory;
the hardware processor configured to execute the software code to:
obtain a reference point metadata associated with a first camera image, the reference point metadata including a timestamp associated with the first camera image;
receive a second camera image from the camera, the second camera image depicting at least one real-world object;
identify at least one reference point depicted in the second camera image based on the reference point metadata including the timestamp associated with the first camera image, the at least one reference point having a predetermined real-world location, the at least one reference point is separate from the at least one real-world object;
map the at least one real-world object to a real-world location of the at least one real-world object based on the predetermined real-world location of the at least one reference point;
merge the second camera image with a virtual object to generate an augmented reality image including the at least one real-world object and the virtual object, wherein a location of the virtual object in the augmented reality image is determined based on the real-world location of the at least one real-world object; and
render the augmented reality image on the display.

15. The mobile user system of claim 14 , wherein the augmented reality image is rendered as a three-hundred-and-sixty degree (360°) image.

16. The mobile user system of claim 14 , wherein the hardware processor is configured to execute the software code to identify the at least one reference point depicted in the second camera image further by performing an image analysis of the second camera image.

17. The mobile user system of claim 14 , wherein the second camera image is captured indoors by the camera.

18. The mobile user system of claim 14 , wherein the mobile user system is one of a smartphone, a tablet computer, or a head-mounted viewing device.

19. The

Assignees

• Disney Entpr Inc

Inventors

• Goslin Michael P
• Chapman Steven M
• Arana Mark

Classifications

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title