Personal protective equipment for navigation and map generation within a hazardous environment using fiducial markers

Patent metadata

Field	Value
Publication number	US-12523477-B2
Application number	US-202118010454-A
Country	US
Kind code	B2
Filing date	Jun 10, 2021
Priority date	Jun 15, 2020
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

The disclosure describes systems of navigating a hazardous environment. The system includes personal protective equipment (PPE) and computing device(s) configured to process sensor data from the PPE, generate pose data of an agent based on the processed sensor data, and track the pose data as the agent moves through the hazardous environment. The PPE may include an inertial measurement device to generate inertial data and a radar device to generate radar data for detecting a presence or arrangement of objects in a visually obscured environment. The PPE may include a thermal image capture device to generate thermal image data for detecting and classifying thermal features of the hazardous environment. The PPE may include one or more sensors to detect a fiducial marker in a visually obscured environment for identifying features in the visually obscured environment. In these ways, the systems may more safely navigate the agent through the hazardous environment.

First claim

What is claimed is:

1 . A system comprising:
a personal protective equipment (PPE) configured to be worn by an agent, wherein the PPE includes a sensor assembly comprising one or more sensors configured to generate sensor data that includes an indication of a fiducial marker in a visually obscured environment; a physical fiducial marker that is placed within the visually obscured environment; and at least one computing device comprising a memory and one or more processors coupled to the memory, wherein the at least one computing device is configured to:
process the sensor data from the sensor assembly to extract fiducial data from the indication of the fiducial marker;
record, in a fiducial datastore, a particular position of the fiducial marker in the visually obscured environment based on one or more inputs from the agent;
associate the fiducial marker with one or more positions in the visually obscured environment based on third-party data;
use additional context surrounding the fiducial marker, as indicated by the third-party data, to determine or refine a position of the fiducial marker within the visually obscured environment;
generate pose data of the agent based on the fiducial data, wherein the pose data includes a location and an orientation of the agent as a function of time; and
track the pose data of the agent as the agent moves through the visually obscured environment.

2 . The system of claim 1 ,
wherein the PPE comprises a breathing apparatus that includes a wearable pack and headgear, and wherein the sensor assembly is integrated within at least one of a frame of the wearable pack, the headgear, or a handheld sensor.

3 . The system of claim 1 , wherein the fiducial data includes a code stored or embodied on the fiducial marker.

4 . The system of claim 1 , wherein the at least one computing device is configured to build, using the fiducial data, a map of the visually obscured environment.

5 . The system of claim 1 , wherein the at least one computing device is further configured to:
identify one or more landmark features based on the fiducial marker, the fiducial marker comprising at least one of an object, a door, a window, a sign, a fiducial, an entry, an exit, a stairway, or a room status; and determine a position of the agent based on the one or more landmark features.

6 . The system of claim 5 , wherein the at least one computing device is configured to determine, based on the one or more landmark features, that a space between the agent and the one or more landmark features is unobscured.

7 . The system of claim 1 ,
wherein the fiducial marker comprises an emissive surface, wherein the sensor assembly comprises a thermal image capture device, and wherein the at least one computing device is configured to detect a pattern or level of emissivity in the emissive surface based on thermal image data.

8 . The system of claim 1 , wherein the fiducial marker comprises a reflective surface configured to reflect electromagnetic radiation corresponding to a pattern.

9 . The system of claim 8 ,
wherein the reflective surface is configured to reflect long-infrared radiation, wherein the sensor assembly comprises a thermal image capture device configured to generate thermal image data from the reflected long-infrared radiation and a thermal energy source configured to emit the long-infrared radiation, and wherein the at least one computing device is configured to detect a pattern in the reflective surface based on thermal image data.

10 . The system of claim 8 ,
wherein the reflective surface is configured to reflect radio waves or microwaves, wherein the sensor assembly comprises a radar device configured to generate radar data from the reflected radio waves or microwaves, and wherein the at least one computing device is configured to detect a pattern in the reflective surface based on the radar data.

11 . The system of claim 8 ,
wherein the reflective surface is configured to reflect long-infrared radiation and radio waves or microwaves, wherein the sensor assembly comprises a thermal image capture device configured to generate thermal image data from the reflected long-infrared radiation, wherein the sensor assembly comprises a radar device configured to generate radar data from the reflected radio waves or microwaves, and wherein the at least one computing device is configured to detect a pattern in the reflective surface based on at least one of the thermal image data or the radar data.

12 . The system of claim 8 , wherein the at least one computing device is configured to operate at least one of a radar device or a thermal image capture device in response to determining that a visible image capture device may not capture visible or infrared light.

13 . The system of claim 1 , wherein the fiducial marker is configured to emit a wireless signal that includes the fiducial data, and wherein the sensor assembly is configured to detect the wireless signal based on a relative proximity to the fiducial marker.

14 . The system of claim 1 , wherein the at least one computing device is further configured to:
generate the pose data based on a relative weighting between the fiducial data and other sensor data; and change the relative weighting based on at least one of a distance or a time of movement of the agent from the fiducial marker.

15 . The system of claim 1 ,
wherein the agent is a first agent, wherein the pose data is first pose data, wherein the fiducial data is first fiducial data, and wherein the at least one computing device is further configured to:
receive second pose data of a second agent, wherein the second pose data includes second fiducial data;
determine that the first fiducial data matches the second fiducial data; and
based on the determination that the first fiducial data matches the second fiducial data, align the one or more landmark features based on the first fiducial data and the second fiducial data to generate map data based on the first and second pose data.

16 . A system comprising:
a personal protective equipment (PPE) configured to be worn by a first agent, wherein the PPE includes a sensor assembly comprising one or more sensors configured to generate sensor data that includes an indication of a fiducial marker in a visually obscured environment; and at least one computing device comprising a memory and one or more processors coupled to the memory, wherein the at least one computing device is configured to: process the sensor data from the sensor assembly to extract first fiducial data from the indication of the fiducial marker; generate first pose data of the agent based on the first fiducial data, wherein the first pose data includes a location and an orientation of the first agent as a function of time; and track the fits pose data of the first agent as the first agent moves through the visually obscured environment; receive second pose data of a second agent, wherein the second pose data includes second fiducial data; determine that the first fiducial data matches the second fiducial data; and based on the determination that the first fiducial data matches the second fiducial data, align the one or more landmark features based on the first fiducial data and the second fiducial data to generate map data based on the first and second pose data.

17 . The system of claim 16 , wherein the first fiducial data and the second fiducial data comprises the same fiducial marker, wherein the at least one computing device is further configured to:
i

Assignees

• 3M Innovative Properties Company

Inventors

• Gabriel Nicholas T
• Kruse John M
• Singh Gautam
• Stankiewicz Brian J
• Aveldson Jason L
• Casner Glenn E
• Hagh-Shenas Haleh
• Herfort Frank T
• Jesme Ronald D
• Lucht Steven G
• Nyland Adam C
• Odom Jacob E
• Tungjunyatham Justin

Classifications

• G06T2207/30204

  Marker · CPC title

• G06T2207/10048

  Infrared image · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B2027/0141

  characterised by the informative content of the display · CPC title