Method for Representing Virtual Information in a View of a Real Environment

Patent metadata

Field	Value
Publication number	US-2024127561-A1
Application number	US-202318538656-A
Country	US
Kind code	A1
Filing date	Dec 13, 2023
Priority date	Oct 12, 2009
Publication date	Apr 18, 2024
Grant date	—

Abstract

A method for representing virtual information in a view of a real environment comprises providing a virtual object having a global position and orientation with respect to a geographic global coordinate system, with first pose data on the global position and orientation of the virtual object, in a database of a server, taking an image of a real environment by a mobile device and providing second pose data as to at which position and with which orientation with respect to the geographic global coordinate system the image was taken. The method further includes displaying the image on a display of the mobile device, accessing the virtual object in the database and positioning the virtual object in the image on the basis of the first and second pose data, manipulating the virtual object or adding a further virtual object, and providing the manipulated virtual object with modified first pose data or the further virtual object with third pose data in the database.

First claim

1 . A method comprising:
receiving an image from a computing device associated with an initial image pose; matching features of a real object depicted in the image to corresponding features of a real object depicted in a reference image having a reference image pose; generating an updated image pose based at last in part on the reference image pose in response to the feature matching; and storing the updated image pose in association with the image.

2 . The method of claim 1 , wherein the initial image pose is determined using a first localization technique, and wherein the reference image pose is determined using a second localization technique.

3 . The method of claim 1 , further comprising:
obtaining a virtual object having an object pose and positioned in the image; determining an updated object pose in accordance with the updated image pose; and storing the virtual object with the updated object pose in network storage.

4 . The method of claim 3 , further comprising:
providing the virtual object and the updated object pose to a second computing device.

5 . The method of claim 4 , wherein the updated object pose is configured to facilitate presentation of the virtual object on a display of the second computing device.

6 . The method of claim 4 , further comprising:
receiving an indication of a manipulated object pose from the virtual object; and storing the manipulated object pose with the virtual object in the network storage.

7 . The method of claim 4 , wherein the virtual object is presented from a first perspective when presented in the image, and wherein the virtual object is presented from a second perspective when presented at the second computing device.

8 . A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:
receive an image from a computing device associated with an initial image pose; match features of a real object depicted in the image to corresponding features of a real object depicted in a reference image having a reference image pose; generate an updated image pose based at last in part on the reference image pose in response to the feature matching; and store the updated image pose in association with the image.

9 . The non-transitory computer readable medium of claim 8 , wherein the initial image pose is determined using a first localization technique, and wherein the reference image pose is determined using a second localization technique.

10 . The non-transitory computer readable medium of claim 8 , further comprising computer readable code to:
obtain a virtual object having an object pose and positioned in the image; determine an updated object pose in accordance with the updated image pose; and store the virtual object with the updated object pose in network storage.

11 . The non-transitory computer readable medium of claim 10 , further comprising computer readable code to:
provide the virtual object and the updated object pose to a second computing device.

12 . The non-transitory computer readable medium of claim 11 , wherein the updated object pose is configured to facilitate presentation of the virtual object on a display of the second computing device.

13 . The non-transitory computer readable medium of claim 11 , further comprising computer readable code to:
receive an indication of a manipulated object pose from the virtual object; and store the manipulated object pose with the virtual object in the network storage.

14 . The non-transitory computer readable medium of claim 11 , wherein the virtual object is presented from a first perspective when presented in the image, and wherein the virtual object is presented from a second perspective when presented at the second computing device.

15 . A system comprising:
one or more processors; and one or more computer readable media comprising computer readable code executable by the one or more processors to:
receive an image from a computing device associated with an initial image pose;
match features of a real object depicted in the image to corresponding features of a real object depicted in a reference image having a reference image pose;
generate an updated image pose based at last in part on the reference image pose in response to the feature matching; and
store the updated image pose in association with the image.

16 . The system of claim 15 , further comprising computer readable code to:
obtain a virtual object having an object pose and positioned in the image; determine an updated object pose in accordance with the updated image pose; and store the virtual object with the updated object pose in network storage.

17 . The system of claim 16 , further comprising computer readable code to:
provide the virtual object and the updated object pose to a second computing device.

18 . The system of claim 17 , wherein the updated object pose is configured to facilitate presentation of the virtual object on a display of the second computing device.

19 . The system of claim 17 , further comprising computer readable code to:
receive an indication of a manipulated object pose from the virtual object; and store the manipulated object pose with the virtual object in the network storage.

20 . The system of claim 17 , wherein the virtual object is presented from a first perspective when presented in the image, and wherein the virtual object is presented from a second perspective when presented at the second computing device.

Assignees

• Apple Inc

Inventors

• Meier Peter
• Kuhn Michael
• Angermann Frank

Classifications

• G01S19/01

  Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T19/00 Primary

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title

• G06T2219/024

  Multi-user, collaborative environment · CPC title