Geographic augmented reality design for low accuracy scenarios

Patent metadata

Field	Value
Publication number	US-11928756-B2
Application number	US-202117482303-A
Country	US
Kind code	B2
Filing date	Sep 22, 2021
Priority date	Sep 22, 2021
Publication date	Mar 12, 2024
Grant date	Mar 12, 2024

Abstract

To present augmented reality features without localizing a user, a client device receives a request for presenting augmented reality features in a camera view of a computing device of the user. Prior to localizing the user, the client device obtains sensor data indicative of a pose of the user, and determines the pose of the user based on the sensor data with a confidence level that exceeds a confidence threshold which indicates a low accuracy state. Then the client device presents one or more augmented reality features in the camera view in accordance with the determined pose of the user while in the low accuracy state.

First claim

What is claimed is:

1. A method for presenting augmented reality features without localizing a user, the method comprising:
receiving, by one or more processors, a request for presenting augmented reality features in a camera view of a computing device of the user;
prior to localizing the user, obtaining, by the one or more processors, sensor data indicative of a location of the user;
determining, by the one or more processors, the location of the user based on the sensor data with a confidence level that exceeds a confidence threshold which indicates a low accuracy state having a low degree of precision for the location of the user;
presenting, by the one or more processors, one or more augmented reality features in the camera view in accordance with the determined location of the user while in the low accuracy state, wherein the one or more augmented reality features have a lower degree of precision than additional or updated augmented reality features presented while in a high accuracy state having a high degree of precision for the location of the user;
after presenting the one or more augmented reality features while in the low accuracy state, localizing the user by determining an updated location of the user with a confidence level within the confidence threshold which indicates the high accuracy state; and
presenting, by the one or more processors, the additional or updated augmented reality features in the camera view in accordance with the updated location of the user while in the high accuracy state.

2. The method of claim 1 , wherein determining the updated location of the user with the confidence level within the confidence threshold includes:
providing, by the one or more processors, an image of the camera view and an indication of the determined location to a server device, wherein the server device compares the image of the camera view to street-level imagery at the determined location to localize the user; and
receiving, by the one or more processors, the updated location of the user from the server device based on the comparison.

3. The method of claim 1 , wherein determining the location of the user includes:
generating, by the one or more processors, a geographic anchor;
generating, by the one or more processors, a visual inertial odometry (VIO) anchor at an initial location of the geographic anchor; and
determining, by the one or more processors, the location of the user based on the VIO anchor.

4. The method of claim 1 , further comprising:
receiving, by the one or more processors, a request for navigation directions to a destination location,
wherein the one or more augmented reality features include an indicator of a direction to travel in to reach the destination location.

5. The method of claim 1 , wherein the location of the user is determined based on the sensor data using a particle filter.

6. The method of claim 1 , wherein obtaining the sensor data includes obtaining the sensor data from at least one of: an accelerometer, a positioning sensor, a transceiver, a gyroscope, a compass, or a magnetometer.

7. The method of claim 1 , wherein the one or more augmented reality features includes an indication of an orientation of a landmark that is not visible within the camera view.

8. A computing device for presenting augmented reality features without localizing a user, the computing device comprising:
a camera;
one or more processors; and
a computer-readable memory coupled to the camera and the one or more processors and storing instructions thereon that, when executed by the one or more processors, cause the computing device to:
receive a request for presenting augmented reality features in a camera view of the camera;
prior to localizing the user, obtain sensor data indicative of a location of the user;
determine the location of the user based on the sensor data with a confidence level that exceeds a confidence threshold which indicates a low accuracy state having a low degree of precision for the location of the user;
present one or more augmented reality features in the camera view in accordance with the determined location of the user while in the low accuracy state, wherein the one or more augmented reality features have a lower degree of precision than augmented reality features presented while in a high accuracy state having a high degree of precision for the location of the user;
after presenting the one or more augmented reality features while in the low accuracy state, localize the user by determining an updated location of the user with a confidence level within the confidence threshold which indicates the high accuracy state; and
present the additional or updated augmented reality features in the camera view in accordance with the updated location of the user while in the high accuracy state.

9. The computing device of claim 8 , wherein to determine the updated location of the user with the confidence level within the confidence threshold, the instructions cause the computing device to:
provide an image of the camera view and an indication of the determined location to a server device, wherein the server device compares the image of the camera view to street-level imagery at the determined location to localize the user; and
receive the updated location of the user from the server device based on the comparison.

10. The computing device of claim 8 , wherein to determine the location of the user, the instructions further cause the computing device to:
generate a geographic anchor;
generate a visual inertial odometry (VIO) anchor at an initial location of the geographic anchor; and
determine the location of the user based on the VIO anchor.

11. The computing device of claim 8 , wherein the instructions further cause the computing device to:
receive a request for navigation directions to a destination location,
wherein the one or more augmented reality features include an indicator of a direction to travel in to reach the destination location.

12. The computing device of claim 8 , wherein the location of the user is determined based on the sensor data using a particle filter.

13. The computing device of claim 8 , wherein the sensor data is obtained from at least one of: an accelerometer, a positioning sensor, a transceiver, a gyroscope, a compass, or a magnetometer.

14. The computing device of claim 8 , wherein the one or more augmented reality features includes an indication of an orientation of a landmark that is not visible within the camera view.

15. A non-transitory computer-readable medium storing instructions thereon that, when executed by one or more processors, cause the one or more processors to:
receive a request for presenting augmented reality features in a camera view of a computing device of a user;
prior to localizing the user, obtain sensor data indicative of a location of the user;
determine the location of the user based on the sensor data with a confidence level that exceeds a confidence threshold which indicates a low accuracy state having a low degree of precision for the location of the user;
present one or more augmented reality features in the camera view in accordance with the determined location of the user while in the low accuracy state, wherein the one or more augmented reality features have a lower degree of precision than augmented reality features presented while in a high accuracy state having a high degree of precision for the location of the user;
after presenting the one or more augmented reality features while in the low accuracy state, localize the user by determining an updated location of the user wit

Assignees

• Google Llc

Inventors

• Mohamed Yousuf Sait Mohamed Suhail
• Le Andre
• Hincapie Juan David
• Ranieri Mirko
• Gorecki Marek
• Zhao Wenli
• Shih Tony
• Zhang Bo
• Sheridan Alan
• Seegmiller Matt

Classifications

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G01C21/20

  Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title