Radio-visual approach for device localization based on a vehicle location

Patent metadata

Field	Value
Publication number	US-11510026-B2
Application number	US-202017128139-A
Country	US
Kind code	B2
Filing date	Dec 20, 2020
Priority date	Jul 1, 2020
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

Disclosed is a radio-visual approach for device localization. In particular, a mobile device may receive, via radio signal(s) emitted by transmitter(s) in a vehicle that is substantially proximate to the mobile device, radio data that represents vehicle information associated with the vehicle, and may then determine or obtain a location of the vehicle in accordance with that vehicle information. Also, the mobile device may capture an image of the vehicle and may use that image as basis for determining a relative location indicating where the mobile device is located relative to the vehicle. Based on the location of the vehicle and on the relative location of the mobile device, the mobile device may then determine a location of the mobile device.

First claim

We claim:

1. A method comprising:
receiving, by one or more processors of a mobile device via one or more radio signals emitted by one or more transmitters in a vehicle that is substantially proximate to the mobile device, radio data that represents vehicle information associated with the vehicle;
determining or obtaining, by the one or more processors, a location of the vehicle in accordance with the vehicle information represented by the radio data;
receiving, by the one or more processors from an image capture device of the mobile device, an image of the vehicle that is substantially proximate to the mobile device;
based at least on the image of the vehicle, determining, by the one or more processors, a relative location indicating where the mobile device is located relative to the vehicle; and
based on the location of the vehicle and on the relative location of the mobile device, determining, by the one or more processors, a location of the mobile device.

2. The method of claim 1 ,
wherein the vehicle information comprises the location of the vehicle, and
wherein determining or obtaining the location of the vehicle is in accordance with the location of the vehicle represented by the radio data.

3. The method of claim 1 ,
wherein the vehicle information comprises one or more parameters associated with the vehicle, and
wherein determining or obtaining the location of the vehicle is in accordance with the one or more parameters associated with the vehicle.

4. The method of claim 1 ,
wherein the mobile device has stored thereon or has access to a database containing location information that indicates respective locations of a plurality of vehicles, and
wherein determining or obtaining the location of the vehicle comprises determining or obtaining the location of the vehicle in accordance with the location information in the database and the vehicle information represented by the radio data.

5. The method of claim 4 ,
wherein the vehicle information comprises one or more parameters associated with the vehicle, and
wherein the location information associates each respective location with one or more respective parameters of a respective vehicle from the plurality of vehicles.

6. The method of claim 4 , wherein the vehicle information comprises one or more parameters associated with the vehicle, the method further comprising:
making a determination, by the one or more processors, that the vehicle information represented by the radio data lacks the location of the vehicle, and
in response to making the determination that the vehicle information represented by the radio data lacks the location of the vehicle, determining or obtaining the location of the vehicle in accordance with the location information in the database and the vehicle information represented by the radio data.

7. The method of claim 1 , wherein determining the relative location of the mobile device comprises applying a computer vision process to determine the relative location based on the image of the vehicle.

8. The method of claim 1 , further comprising:
in response to receiving radio data that represents vehicle information associated with the vehicle, causing, by the one or more processors, an output device of the mobile device to output a notification, wherein the notification requests capture of the image of the vehicle that is substantially proximate to the mobile device.

9. The method of claim 8 ,
wherein the output device is a display device of the mobile device, and
wherein outputting the notification comprises outputting the notification via a graphical user interface (GUI) displayed by the display device.

10. The method of claim 9 , wherein the GUI is of a navigation application.

11. The method of claim 9 , wherein the notification comprises one or more of (i) a representation of a field of view of the image capture device of the mobile device, (ii) information about the vehicle, or (iii) a textual description requesting capture of the image of the vehicle that is substantially proximate to the mobile device.

12. The method of claim 1 , further comprising:
performing, by the one or more processors, one or more of: (i) causing the mobile device to provide the determined location of the mobile device to an entity or (ii) using the determined location of the mobile device as basis for one or more operations executed by the mobile device.

13. The method of claim 12 , wherein the performing comprises using or causing using of the determined location as a reference location in a position determination process.

14. The method of claim 1 , further comprising:
determining or obtaining, by the one or more processors, an orientation of the vehicle in accordance with the vehicle information represented by the radio data;
based at least on the image of the vehicle, determining, by the one or more processors, a relative orientation indicating how the mobile device is oriented relative to the vehicle; and
based at least on the orientation of the vehicle and on the relative orientation of the mobile device, determining, by the one or more processors, one or more of an orientation or heading of the mobile device.

15. The method of claim 14 , further comprising:
performing, by the one or more processors, one or more of: (i) causing the mobile device to provide the determined orientation of the mobile device to an entity or (ii) using the determined orientation of the mobile device as basis for one or more operations executed by the mobile device.

16. The method of claim 15 , wherein the performing comprises using or causing using of the determined orientation as a reference orientation in a position determination process.

17. An apparatus comprising:
one or more processors;
a non-transitory computer readable medium; and
program instructions stored on the non-transitory computer readable medium and executable by the one or more processors to:
receive, via one or more radio signals emitted by one or more transmitters in a vehicle that is substantially proximate to a mobile device, radio data that represents vehicle information associated with the vehicle;
determine or obtain a location of the vehicle in accordance with the vehicle information represented by the radio data;
receive, from an image capture device of the mobile device, an image of the vehicle that is substantially proximate to the mobile device;
based at least on the image of the vehicle, determine a relative location indicating where the mobile device is located relative to the vehicle; and
based on the location of the vehicle and on the relative location of the mobile device, determine a location of the mobile device.

18. The apparatus of claim 17 , wherein the apparatus takes a form of or includes the mobile device.

19. The apparatus of claim 17 , wherein the program instructions are further executable to perform one or more of: (i) causing the mobile device to provide the determined location of the mobile device to an entity or (ii) using the determined location of the mobile device as basis for one or more operations executed by the mobile device.

20. A non-transitory computer readable medium having stored thereon instructions executable by one or more processors to cause a mobile device to perform operations comprising:
receiving, via one or more radio signals emitted by one or more transmitters in a vehicle that is substantially proximate to the mobile device, radio data that represents vehicle information associated with the vehicle;
determi

Assignees

• Here Global Bv

Inventors

• Ivanov Pavel
• Krasniansky Michael

Classifications

• G01S2205/01

  specially adapted for specific applications · CPC title

• H04W4/023 Primary

  using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds · CPC title

• G01S1/047

  Displays or indicators (G01S1/06 takes precedence) · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G01S5/16

  using electromagnetic waves other than radio waves · CPC title