Mobile device navigation system

Patent metadata

Field	Value
Publication number	US-11943679-B2
Application number	US-202217861167-A
Country	US
Kind code	B2
Filing date	Jul 8, 2022
Priority date	Sep 19, 2019
Publication date	Mar 26, 2024
Grant date	Mar 26, 2024

Abstract

Location mapping and navigation user interfaces may be generated and presented via mobile computing devices. A mobile device may detect its location and orientation using internal systems, and may capture image data using a device camera. The mobile device also may retrieve map information from a map server corresponding to the current location of the device. Using the image data captured at the device, the current location data, and the corresponding local map information, the mobile device may determine or update a current orientation reading for the device. Location errors and updated location data also may be determined for the device, and a map user interface may be generated and displayed on the mobile device using the updated device orientation and/or location data.

First claim

What is claimed is:

1. A method, comprising:
determining, by a computing device, a current location of the computing device;
receiving, by the computing device, image data corresponding to one or more images captured by one or more cameras at the current location of the computing device;
determining one or more physical objects located near the current location from map information corresponding to the current location of the computing device;
matching one or more objects captured within the image data with the map information, wherein matching the one or more objects captured within the image data comprises matching the one or more objects captured within the image data with the one or more physical objects located near the current location; and
determining, by the computing device, a directional orientation of the computing device based at least in part on matching the one or more objects captured within the image data with the map information, wherein determining the directional orientation comprises determining the directional orientation based at least in part on matching the one or more objects captured within the image data with the one or more physical objects located near the current location.

2. The method of claim 1 , wherein the one or more cameras comprise one or more cameras of the computing device.

3. The method of claim 1 , wherein receiving the image data comprises:
causing the one or more images to be captured by the one or more cameras in response to a mobile application being opened or launched.

4. The method of claim 1 , further comprising:
causing one or more other devices to capture at least a portion of the one or more images, and wherein receiving the image data comprises receiving the image data from the one or more other devices.

5. The method of claim 1 , wherein matching the one or more objects captured within the image data with the map information comprises:
identifying one or more line segments within the image data; and
determining the one or more objects within the image data based on the one or more line segments.

6. The method of claim 5 , wherein determining the one or more objects comprises determining one or more streets, sidewalks, or paths within the image data, and wherein matching the one or more objects captured within the image data with the map information comprises matching the one or more streets, sidewalks, or paths within the image data with one or more streets, sidewalks, or paths represented within the map information.

7. The method of claim 1 , wherein matching the one or more objects captured within the image data with the map information comprises:
identifying one or more buildings or landmarks within the image data as at least a portion of the one or more objects; and
matching the one or more buildings or landmarks within the image data with one or more buildings or landmarks represented within the map information.

8. The method of claim 1 , wherein determining the current location of the computing device comprises matching the one or more objects captured within the image data with the map information to determine the current location of the computing device.

9. One or more non-transitory, computer-readable media having instructions stored thereon, wherein the instructions, when executed by one or more processors, cause the one or more processors to:
determine a current location of a computing device;
receive image data corresponding to one or more images captured by one or more cameras at the current location of the computing device;
determine one or more physical objects located near the current location from map information corresponding to the current location of the computing device;
match one or more objects captured within the image data with the map information, wherein to match the one or more objects captured within the image data with the map information comprises to match the one or more objects captured within the image data with the one or more physical objects located near the current location; and
determine a directional orientation of the computing device based at least in part on matching the one or more objects captured within the image data with the map information, wherein to determine the directional orientation comprises to determine the directional orientation based at least in part on matching the one or more objects captured within the image data with the one or more physical objects located near the current location.

10. The one or more non-transitory, computer-readable media of claim 9 , wherein to receive the image data comprises to:
cause the one or more images to be captured by the one or more cameras in response to a mobile navigation application being opened or launched.

11. The one or more non-transitory, computer-readable media of claim 9 , wherein to match the one or more objects captured within the image data with the map information comprises to:
identify one or more line segments within the image data; and
determine the one or more objects within the image data based on the one or more line segments.

12. The one or more non-transitory, computer-readable media of claim 9 , wherein to match the one or more objects captured within the image data with the map information comprises to:
identify one or more buildings or landmarks within the image data as at least a portion of the one or more objects; and
match the one or more buildings or landmarks within the image data with one or more buildings or landmarks represented within the map information.

13. A computing device, comprising:
memory to store image data captured by one or more cameras; and
one or more processors coupled to the memory, the one or more processors to:
determine a current location of the computing device;
receive the image data corresponding to one or more images captured by the one or more cameras at the current location of the computing device;
determine one or more physical objects located near the current location from map information corresponding to the current location of the computing device;
match one or more objects captured within the image data with the map information, wherein to match the one or more objects captured within the image data with the map information comprises to match the one or more objects captured within the image data with the one or more physical objects located near the current location; and
determine a directional orientation of the computing device based at least in part on matching the one or more objects captured within the image data with the map information, wherein to determine the directional orientation comprises to determine the directional orientation based at least in part on matching the one or more objects captured within the image data with the one or more physical objects located near the current location.

14. The computing device of claim 13 , wherein to match the one or more objects within the image data with the map information comprises to:
identify one or more buildings or landmarks within the image data as at least a portion of the one or more objects; and
match the one or more buildings or landmarks within the image data with one or more buildings or landmarks represented within the map information.

Assignees

• Apple Inc

Inventors

• Mayor Robert William
• Miller Isaac T
• Howell Adam S
• Majjigi Vinay R
• Ruepp Oliver
• Ulbricht Daniel
• Naroditsky Oleg
• Lipski Christian
• Cier Sean P
• Bae Hyojoon
• Godha Saurabh
• Coleman Patrick J

Classifications

• G01C21/30

  Map- or contour-matching · CPC title

• G01C21/3647 Primary

  Guidance involving output of stored or live camera images or video streams · CPC title

• H04W4/024 Primary

  Guidance services · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title