Path visualization for motion planning

Patent metadata

Field	Value
Publication number	US-2017193705-A1
Application number	US-201615396109-A
Country	US
Kind code	A1
Filing date	Dec 30, 2016
Priority date	Dec 31, 2015
Publication date	Jul 6, 2017
Grant date	—

Abstract

An augmented reality device includes one or more sensors for imaging and/or detecting an environment and a transparent display for displaying virtual objects. The augmented reality device monitors various biometric attributes of a user and determines the user's location within the environment. The augmented reality device determines a virtual path from the user's location to a selected destination within the environment using the monitored biometric attributes as one or more constraints in the pathfinding determination. The determined virtual path is displayed on the transparent display as a virtual object such that it appears overlaid on the user's environment. The augmented reality device monitors the user as he or she traverses the virtual path through the environment, and updates the virtual path in response to changes in one or more of the user's monitored biometric attributes.

First claim

We claim:

1 . A method for determining and displaying a virtual path using augmented reality, the method comprising:
receiving, with at least one hardware processor, sensor data about an environment from at least one sensor communicatively coupled to the at least one hardware processor; determining, with the at least one hardware processor, a terrain type for the environment based on the sensor data; determining, with the at least one hardware processor, a virtual path using the sensor data and the determined terrain type; and displaying, on a display communicatively coupled to the at least one hardware processor, the virtual path, wherein the virtual path appears overlaid on corresponding portions of the environment when viewed through the display.

2 . The method of claim 1 , further comprising:
monitoring, with the at least one hardware processor, biometric data for a user using at least one biometric sensor communicatively coupled to the at least one hardware processor; and wherein determining the virtual path comprises determining the virtual path based on the monitored biometric data.

3 . The method of claim 2 , further comprising:
receiving user-provided health-related information about the user; and wherein determining the virtual path comprises determining the virtual path based on the user-provided health-related information.

4 . The method of claim 1 , further comprising:
recognizing an object in the environment; and wherein determining the virtual path further comprises determining the virtual path based on the recognized object.

5 . The method of claim 1 , further comprising:
identifying a portion of the environment as being unsafe; and wherein determining the virtual path further comprises determining the virtual path to circumnavigate the identified portion of the unsafe environment.

6 . The method of claim 1 , further comprising:
detecting an obstacle intersecting the determined virtual path; modifying terrain data associated with the determined virtual path to account for the detected obstacle; re-determining the determined virtual path to circumnavigate the detected obstacle based on the modified terrain data; and displaying the re-determined virtual path.

7 . The method of claim 1 , further comprising:
monitoring, with the at least one hardware processor, biometric data for a user using at least one biometric sensor communicatively coupled to the at least one hardware processor; determining that the monitored biometric data meets or exceeds one or more biometric safety threshold values; in response to the determination that the monitored biometric data meets or exceeds the one or more biometric safety threshold values, determining a plurality of virtual paths, wherein each of the virtual paths selected from the virtual paths are associated with a different set of biometric safety threshold values; presenting an option to select at least one of the virtual paths of the plurality of virtual paths as the virtual path to use instead of the determined virtual path; and in response to a selection of the at least one virtual path, displaying the selected at least one virtual path.

8 . A system for determining and displaying a virtual path using augmented reality, the system comprising:
a computer-readable memory storing computer-executable instructions thereon; and at least one hardware processor communicatively coupled to the computer-readable memory that, when the computer-executable instructions are executed, configures the system to:
receive sensor data about an environment from at least one sensor communicatively coupled to the at least one hardware processor;
determine a terrain type for the environment based on the sensor data;
determine a virtual path using the sensor data and the determined terrain type; and
display, on a display communicatively coupled to the at least one hardware processor, the virtual path, wherein the virtual path appears overlaid on corresponding portions of the environment when viewed through the display.

9 . The system of claim 8 , wherein the system is further configured to:
monitor, with the at least one hardware processor, biometric data for a user using at least one biometric sensor communicatively coupled to the at least one hardware processor; and wherein the determination of the virtual path comprises determining the virtual path based on the monitored biometric data.

10 . The system of claim 9 , wherein the system is further configured to:
receive user-provided health-related information about the user; and wherein the determination of the virtual path comprises determining the virtual path based on the user-provided health-related information.

11 . The system of claim 8 , wherein the system is further configured to:
recognize an object in the environment; and wherein the determination of the virtual path further comprises determining the virtual path based on the recognized object.

12 . The system of claim 8 , further wherein the system is further configured to:
identify a portion of the environment as being unsafe; and wherein the determination of the virtual path further comprises determining the virtual path to circumnavigate the identified portion of the unsafe environment.

13 . The system of claim 8 , wherein the system is further configured to:
detect an obstacle intersecting the determined virtual path; modify terrain data associated with the determined virtual path to account for the detected obstacle; re-determine the determined virtual path to circumnavigate the detected obstacle based on the modified terrain data; and display the re-determined virtual path.

14 . The system of claim 8 , wherein the system is further configured to:
monitor, with the at least one hardware processor, biometric data for a user using at least one biometric sensor communicatively coupled to the at least one hardware processor; determine that the monitored biometric data meets or exceeds one or more biometric safety threshold values; in response to the determination that the monitored biometric data meets or exceeds the one or more biometric safety threshold values, determine a plurality of virtual paths, wherein each of the virtual paths selected from the virtual paths are associated with a different set of biometric safety threshold values; present an option to select at least one of the virtual paths of the plurality of virtual paths as the virtual path to use instead of the determined virtual path; and in response to a selection of the at least one virtual path, display the selected at least one virtual path.

15 . A computer-readable medium storing computer-executable instructions thereon that, when executed by at least one hardware processor, causes a system to perform a plurality of operations, the plurality of operations comprising:
receiving sensor data about an environment from at least one sensor communicatively coupled to the at least one hardware processor; determining, a terrain type for the environment based on the sensor data; determining a virtual path using the sensor data and the determined terrain type; and displaying, on the display communicatively coupled to the at least one hardware processor, the virtual path, wherein the virtual path appears overlaid on corresponding portions of the environment when viewed through the display.

16 . The computer-readable medium of claim 15 , wherein the plurality of operations further comprises:
monitoring, with the at least one hardware processor, biometric data for a user using at least one bi

Assignees

• Daqri Llc

Inventors

• Mullins Brian
• Kammerait Matthew
• Irving Jr Frank Chester

Classifications

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G01C21/20

  Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title