Spatial localization design service

Patent metadata

Field	Value
Publication number	US-12340567-B2
Application number	US-202318498024-A
Country	US
Kind code	B2
Filing date	Oct 30, 2023
Priority date	May 8, 2018
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

A synthetic world interface may be used to model digital environments, sensors, and motions for the evaluation, development, and improvement of localization algorithms. A synthetic data cloud service with a library of sensor primitives, motion generators, and environments with procedural and game-like capabilities, facilitates engineering design for a manufactural solution that has localization capabilities. In some embodiments, a sensor platform simulator operates with a motion orchestrator, an environment orchestrator, an experiment generator, and an experiment runner to test various candidate hardware configurations and localization algorithms in a virtual environment, advantageously speeding development and reducing cost. Thus, examples disclosed herein may relate to virtual reality (VR) or mixed reality (MR) implementations.

First claim

What is claimed is:

1. A system comprising:
a memory embodied with executable instructions; and
a processor programmed for:
simulating a virtual environment;
determining disparity data of a localization algorithm tested against ground truth data in the simulated virtual environment from the localization algorithm tested against a first hardware configuration in the simulated virtual environment;
determining that the disparity data exceeds a variance threshold;
directing an artificial intelligence (AI) application to run subsequent testing of the localization algorithm against a second hardware configuration, different from the first hardware configuration, in the simulated virtual environment; and
modifying the localization algorithm based on the subsequent testing.

2. The system of claim 1 , wherein the processor is further programmed for determining, based on the first hardware configuration, localization parameters including coordinates of a virtual camera in the simulated virtual environment.

3. The system of claim 1 , wherein the processor is further programmed for selecting the modified localization algorithm for the first hardware configuration.

4. The system of claim 1 , wherein the processor is further programmed for:
simulating motion of the first hardware configuration within the simulated virtual environment, and
applying the localization algorithm to the first hardware configuration having the simulated motion within the simulated virtual environment.

5. The system of claim 4 , wherein the processor is further programmed for:
generating synthetic experiment data for the first hardware configuration having the simulated motion.

6. The system of claim 5 , wherein the synthetic experiment data comprises inertial measurement unit (IMU) data, the IMU data including one or more of: accelerometer data, gyroscope data, and magnetometer data.

7. The system of claim 1 , wherein the first hardware configuration includes a first camera and the second hardware configuration includes a second camera that is different from the first camera.

8. A computerized method comprising:
simulating a virtual environment;
determining disparity data of a localization algorithm tested against ground truth data in the simulated virtual environment from the localization algorithm tested against a first hardware configuration in the simulated virtual environment;
determining that the disparity data exceeds a variance threshold;
directing an artificial intelligence (AI) application to run subsequent testing of the localization algorithm against a second hardware configuration, different from the first hardware configuration, in the simulated virtual environment; and
modifying the localization algorithm based on the subsequent testing.

9. The method of claim 8 , further comprising determining, based on the first hardware configuration, localization parameters including coordinates of a virtual camera in the simulated virtual environment.

10. The method of claim 8 , further comprising selecting the modified localization algorithm for the first hardware configuration.

11. The method of claim 8 , further comprising:
simulating motion of the first hardware configuration within the simulated virtual environment, and
applying the localization algorithm to the first hardware configuration having the simulated motion within the simulated virtual environment.

12. The method of claim 11 , further comprising:
generating synthetic experiment data for the first hardware configuration having the simulated motion.

13. The method of claim 12 , wherein the synthetic experiment data comprises inertial measurement unit (IMU) data, the IMU data including one or more of: accelerometer data, gyroscope data, and magnetometer data.

14. The method of claim 8 , wherein the first hardware configuration includes a first camera and the second hardware configuration includes a second camera that is different from the first camera.

15. A computer storage device having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:
simulating a virtual environment;
determining disparity data of a localization algorithm tested against ground truth data in the simulated virtual environment from the localization algorithm tested against a first hardware configuration in the simulated virtual environment;
determining that the disparity data exceeds a variance threshold;
directing an artificial intelligence (AI) application to run subsequent testing of the localization algorithm against a second hardware configuration, different from the first hardware configuration, in the simulated virtual environment; and
modifying the localization algorithm based on the subsequent testing.

16. The computer storage device of claim 15 , wherein the operations further comprise determining, based on the first hardware configuration, localization parameters including coordinates of a virtual camera in the simulated virtual environment.

17. The computer storage device of claim 15 , wherein the operations further comprise selecting the modified localization algorithm for the first hardware configuration.

18. The computer storage device of claim 15 , wherein the operations further comprise:
simulating motion of the first hardware configuration within the simulated virtual environment, and
applying the localization algorithm to the first hardware configuration having the simulated motion within the simulated virtual environment.

19. The computer storage device of claim 18 , wherein the operations further comprise:
generating synthetic experiment data for the first hardware configuration having the simulated motion.

20. The computer storage device of claim 19 , wherein the synthetic experiment data comprises inertial measurement unit (IMU) data, the IMU data including one or more of: accelerometer data, gyroscope data, and magnetometer data.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Ebstyne Michael
• Escos Pedro Urbina
• Shalev Emanuel
• Kipman Alex
• Pekelny Yuri
• Chan Jonathan Chi Hang

Classifications

• G06F30/27

  using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F11/261

  by simulating additional hardware, e.g. fault simulation · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title