Threat identification system

Patent metadata

Field	Value
Publication number	US-2016342840-A1
Application number	US-201615158498-A
Country	US
Kind code	A1
Filing date	May 18, 2016
Priority date	May 18, 2015
Publication date	Nov 24, 2016
Grant date	—

Abstract

A head mounted device (HMD) includes a transparent display, sensors to generate sensor data, and a processor. The processor identifies a threat condition based on a threat pattern and the sensor data, and generates a warning notification in response to the identified threat condition. The threat pattern includes preconfigured thresholds for the sensor data. The HMD displays AR content comprising the warning notification in the transparent display.

First claim

1 . A head mounted device (HMD) comprising:
a transparent display; a plurality of sensors configured to measure sensor data; and a processor comprising an augmented reality (AR) application and a threat application, the threat application being configured to identify a threat condition based on a threat pattern and the sensor data, and to generate a warning notification in response to the identified threat condition, the threat pattern comprising preconfigured thresholds for the sensor data, the threat application comprising a threat learning module being configured to access historical sensor data corresponding to user tasks, to identify user tasks with a negative outcome, to filter historical sensor data resulting in the negative outcome from the historical sensor data, and to generate the threat pattern based on the filtered historical sensor data; and the AR application being configured to cause a display of AR content comprising the warning notification in the transparent display.

2 . The HMD of claim 1 , wherein the threat application comprises:
the threat learning module configured to receive the threat pattern from a server, the server configured to generate the preconfigured thresholds for the sensor data; and a threat identification module configured to identify one of the sensor data transgressing one of the preconfigured thresholds, and to generate correction information based on the identified sensor data exceeding one of the preconfigured thresholds, the correction information including instructions to address the threat condition.

3 . (canceled)

4 . The HMD of claim 1 , wherein the threat learning module is configured to identify the negative outcome based on preconfigured parameters for the sensor data.

5 . The HMD of claim 2 , wherein the AR content includes the correction information, the correction information including instructions to a user of the HMD and a three-dimensional model including the correction information.

6 . The HMD of claim 1 , wherein the plurality of sensors includes:
a first set of sensors configured to generate HMD-based sensor data related to the HMD; and a second set of sensors configured to generate ambient-based sensor data related to an ambient environment of the HMD, the ambient environment including a physical space within a predefined distance of the HMD, the sensor data comprising the HMD-based sensor data and the ambient-based sensor data.

7 . The HMD of claim 6 , wherein the threat pattern is generated based on preconfigured parameters for the HMD-based sensor data and the ambient-based sensor data.

8 . The HMD of claim 6 ,
wherein the threat learning module is configured to identify a task provided by the AR application, a physical object associated with the task, historical HMD-based sensor data associated with the task and the physical object, historical ambient-based sensor data associated with the task and the physical object, historical user activities associated with the task, and an outcome of the historical user activities, wherein the threat learning module is configured to generate the threat pattern based on a negative outcome of the historical user activities, the negative outcome being determined in response to the historical HMD-based sensor data or historical ambient-based sensor data transgressing a predefined threshold, the threat pattern comprising a series of user activities with respect to the physical object, a series of HMD-based sensor data corresponding to the series of user activities, and a series of historical ambient-based sensor data corresponding to the series of user activities.

9 . The HMD of claim 6 , wherein the threat application is configured to receive further ambient-based sensor data from a third set of sensors external to the HMD,
the threat pattern comprising a series of historical ambient-based sensor data from the third set of sensors.

10 . The HMD of claim 1 , wherein the AR application is configured to determine a travel direction of the HMD based on at least one of the plurality of sensors, to determine that the travel direction is outside a field of view of a user of the HMD, to identify a moving physical object outside a field of view of the user of the HMD, to determine a trajectory of the moving physical object based on a travel path of the moving physical object, to determine that the trajectory of the moving physical object intersects the travel direction of the HMD at an estimated intersection located within a threshold distance of the HMD, and to generate the warning notification based on the estimated intersection.

11 . A method comprising:
measuring sensor data using a plurality of sensors in a head-mounted device (HMD); identifying a threat condition based on a comparison of a threat pattern with the sensor data, the threat pattern comprising preconfigured thresholds for the sensor data; accessing historical sensor data corresponding to user tasks; identifying user tasks with a negative outcome; filtering historical sensor data resulting in the negative outcome from the historical sensor data, the threat pattern generated based on the filtered historical sensor data; generating the threat pattern based on the filtered historical sensor data; generating a warning notification in response to the identified threat condition; and displaying, using a hardware processor of the HMD, AR content including the warning notification in a transparent display of the HMD.

12 . The method of claim 11 , further comprising:
receiving the threat pattern from a server, the server configured to generate the preconfigured thresholds for the sensor data; and identifying one of the sensor data transgressing one of the preconfigured thresholds; and generating correction information based on the identified sensor data exceeding one of the preconfigured thresholds, the correction information including instructions to address the threat condition.

13 . (canceled)

14 . The method of claim 11 , further comprising:
identifying the negative outcome based on preconfigured parameters for the sensor data.

15 . The method of claim 12 , wherein the AR content includes the correction information, the correction information including instructions to a user of the HMD and a three-dimensional model including the correction information.

16 . The method of claim 11 , wherein the plurality of sensors includes:
a first set of sensors being configured to generate HMD-based sensor data related to the HMD; and a second set of sensors being configured to generate ambient-based sensor data related to an ambient environment of the HMD, the ambient environment including a physical space within a predefined distance of the HMD, the sensor data comprising the HMD-based sensor data and the ambient-based sensor data.

17 . The method of claim 16 , wherein the threat pattern is generated based on preconfigured parameters for the HMD-based sensor data and the ambient-based sensor data.

18 . The method of claim 16 , further comprising:
identifying a task provided by an AR application, a physical object associated with the task, historical HMD-based sensor data associated with the task and the physical object, historical ambient-based sensor data associated with the task and the physical object, historical user activities associated with the task, and an outcome of the historical user activities; generating the threat pattern based on a negative outcome of the historical user activities, the negative outcome being determined in response to the histor

Assignees

• Daqri Llc

Inventors

• Mullins Brian
• Kammerait Matthew

Classifications

• G06V10/803

  of input or preprocessed data · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F18/251

  of input or preprocessed data · CPC title

• G06F3/14

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title