Augmented reality safety automation zone system and method

Patent metadata

Field	Value
Publication number	US-10445944-B2
Application number	US-201816165360-A
Country	US
Kind code	B2
Filing date	Oct 19, 2018
Priority date	Nov 13, 2017
Publication date	Oct 15, 2019
Grant date	Oct 15, 2019

Abstract

An industrial visualization system generates and delivers virtual reality (VR) and augmented reality (AR) presentations of industrial facilities to wearable appliances to facilitate remote or enhanced interaction with automation systems within the facility. The presentation system can also leverage safety zone definition data to visualize graphical representations of defined safety zones within a line of sight of the wearable appliance. The safety zones can be rendered as three-dimensional semi-transparent overlays corresponding to the defined dimensions and location of the safety zone relative to its corresponding safety sensor. By visualizing defined safety zones as overlays within a wearer's field of view as part of an AR presentation, the presentation system can serve as a visual guide that helps the wearer to avoid inadvertently entering the safety zones, thereby reducing machine downtime due to accidental initiation of safety actions.

First claim

What is claimed is:

1. A system, comprising:
a memory that stores executable components;
a processor, operatively coupled to the memory, that executes the executable components, the executable components comprising:
a device interface component configured to receive industrial data from industrial devices associated with an industrial facility, the industrial devices comprising at least an industrial controller that receives input data from at least one industrial input device and sends output data to one or more industrial output devices;
a client interface component configured to receive user location data and user orientation data from a client device, wherein the user location data and the user orientation data indicate a current location and orientation, respectively, of the client device; and
a rendering component configured to generate augmented reality presentation data that renders an augmented reality presentation of the industrial facility on a wearable appliance,
wherein
the device interface component is further configured to receive safety zone configuration data from a safety sensor device, the safety zone configuration data defining dimensions of a safety field defined for the safety sensor device, and
the rendering component is further configured to, in response to determining that the user location data and the user orientation data indicate that a field of view of the client device encompasses a portion of the safety field, render, on the augmented reality presentation based on the safety zone configuration data, a graphical representation of the portion of the safety field.

2. The system of claim 1 , wherein the device interface component is further configured to receive safety sensor location data indicating a location of the safety sensor device and safety sensor orientation data indicating an orientation of the safety sensor device, and the rendering component is further configured to render the graphical representation of the portion of the safety field based on the safety zone configuration data, the safety sensor location data, and the safety sensor orientation data correlated with the user location data and the user orientation data.

3. The system of claim 2 , wherein the graphical representation of the portion of the safety field comprises a semi-transparent three-dimensional shape rendered at a location within the field of view corresponding to the location of the safety sensor device indicated by the safety sensor location data.

4. The system of claim 2 , wherein the rendering component is further configured to, in response to a change to at least one of the safety sensor location data or the safety sensor orientation data, update at least one of a location of the graphical representation within the field of view or an orientation of the graphical representation within the field of view in accordance with the change.

5. The system of claim 2 , wherein the client interface component is further configured to determine whether the user location data corresponds to a location within the safety field based on a correlation of the user location data, the safety zone configuration data, the safety sensor location data, and the safety sensor orientation data.

6. The system of claim 5 , wherein the device interface component is further configured to, in response to a determination by the client interface component that the user location data corresponds to the location within the safety field and that a defined safety function has not been performed by the safety sensor device, generate a control output that places an industrial system in a safe state.

7. The system of claim 5 , wherein
the safety zone configuration data further defines dimensions of one or more warning fields defined for the safety sensor device, and
the client interface component is further configured to, in response to determining that the user location data corresponds to a location within a warning field of the one or more warning fields, send at least one of a visual indication, an audible indication, or a tactile indication to the client device.

8. The system of claim 1 , wherein the safety zone configuration data defines dimensions of multiple safety fields defined for the safety sensor device, and the graphical representation renders the multiple safety fields using respective different colors.

9. The system of claim 5 , wherein the device interface component is further configured to, in response to a determination by the client interface component that the user location data corresponds to the location within the safety field, increase a brightness of the graphical representation of the portion of the safety field.

10. The system of claim 1 , wherein the client device is at least one of a wearable appliance or a hand-held client device.

11. A method, comprising:
receiving, by a system comprising a processor, industrial data generated by industrial devices of an industrial facility, the industrial devices comprising at least an industrial controller that receives input data from at least one industrial input device and sends output data to one or more industrial output devices;
receiving, by the system, user location data and user orientation data from a wearable appliance, wherein the user location data and the user orientation data indicate a current location and orientation, respectively, of the wearable appliance;
generating, by the system, augmented reality presentation data that renders an augmented reality presentation of the industrial facility on a wearable appliance;
receiving, by the system, safety zone configuration data from a safety sensor device, the safety zone configuration data defining dimensions of a safety field defined for the safety sensor device; and
in response to determining that the user location data and the user orientation data indicate that a portion of the safety field is within a field of view of the wearable appliance, rendering, by the system on the augmented reality presentation based on the safety zone configuration data, a graphical representation of the portion of the safety field.

12. The method of claim 11 , further comprising:
receiving, by the system, safety sensor location data indicating a location of the safety sensor device and safety sensor orientation data indicating an orientation of the safety sensor device; and
rendering, by the system, the graphical representation of the portion of the safety field based on the safety zone configuration data, the safety sensor location data, and the safety sensor orientation data correlated with the user location data and the user orientation data.

13. The method of claim 12 , wherein the rendering the graphical representation of the portion of the safety field comprises rendering the graphical representation of the portion of the safety field as a semi-transparent three-dimensional shape rendered at a location within the field of view corresponding to the location of the safety sensor device indicated by the safety sensor location data.

14. The method of claim 12 , further comprising:
in response to detecting a change to at least one of the safety sensor location data or the safety sensor orientation data, updating, by the system, at least one of a location of the graphical representation within the field of view or an orientation of the graphical representation within the field of view in accordance with the change.

15. The method of claim 12 , further comprising determining, by the system based on a correlation of the user location data, the safety zone configuration data, the safety sensor location data, and the safety se

Assignees

• Rockwell Automation Tech Inc

Inventors

• Galera Richard
• Pritchard John

Classifications

• G06T2210/62

  Semi-transparency · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G05B2219/50193

  Safety in general · CPC title

• G05B19/4061

  Avoiding collision or forbidden zones · CPC title

• G05B2219/32014

  Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command · CPC title