Method and apparatus for discreet person identification on pocket-size offline mobile platform with augmented reality feedback with real-time training capability for usage by universal users

Patent metadata

Field	Value
Publication number	US-12094243-B2
Application number	US-202117324909-A
Country	US
Kind code	B2
Filing date	May 19, 2021
Priority date	May 19, 2020
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

Embodiments may provide improved techniques and devices for person identification that are discreet, offline, that can be trained in real-time and are physically small to provide the comfortable and convenient use in everyday situations. For example, in an embodiment, a system may comprise a wearable device comprising a visual display, an audio output device, and at least one sensor, a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, receiving, from at least one sensor, data captured by the sensor, identifying a person based on the data captured by the sensor using a trained machine learning model, and indicating an identity of the person using at least one of the visual display and the audio output device. A person is also identified using fusion data recognition of sensor data fusion of at least two types of sensor data. System can perform person identification in total darkness using appropriate senor data identification. The system can train an un-trained machine learning model in real-time upon user request and/or command using the data collected by the sensors.

First claim

What is claimed is:

1. A system comprising:
a wearable device comprising a visual display, an audio output device, and at least one sensor;
a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor;
receiving, from the at least one sensor, data captured by the sensor;
identifying a person based on the data captured by the sensor using a trained machine learning model; and
indicating an identity of the person using at least one of the visual display and the audio output device, wherein the wearable device comprises glasses, the sensors comprise at least one of a camera, a microphone, a body odor sensor, and a thermal imaging camera, and wherein the data captured by the sensor and the person is identified from at least one of the groupings consisting of:
the data captured by the sensor is video and/or image data and the person is identified using gait recognition;
the data captured by the sensor is image data and the person is identified using periocular and/or iris recognition;
the data captured by the sensor is image data and the person is identified using thermal image recognition;
the data captured by the sensor is audio data and the person is identified using voice recognition;
the data captured by the sensor is body odor data and the person is identified using body odor recognition;
the data captured by the sensor is at least one of body odor data and thermal image data and the person is identified using body odor recognition or using thermal image recognition in at least one of reduced light, night time, total darkness and total silence;
the data captured by the sensors are facial image data, voice data, thermal image data, periocular image data, iris image data, body odor data and gait data and the person is identified using fusion data recognition of sensor data fusion of at least two different types of the sensor data; and
the data captured by the sensors are facial image data, thermal image data, periocular image data and iris image data, and the person is identified using fusion data recognition.

2. The system of claim 1 , wherein the data captured by the sensor is image data and the person is identified using facial recognition.

3. The system of claim 1 , wherein the facial recognition is performed in real-time and the machine learning model is one of OpenCV Caffe, OpenFace, FaceNet or any other openly available or custom libraries and or models or software.

4. The system of claim 1 , wherein the computer system is contained in a device separate from the glasses.

5. The system of claim 4 , wherein the computer system is configured to communicate with the glasses using wireless communications.

6. The system of claim 1 , wherein the computer system is contained in the glasses.

7. The system of claim 1 , wherein the system is configured to function without connection to or communication with any other computing device.

8. A system comprising:
a wearable device comprising a visual display, an audio output device, and at least one sensor;
a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor;
receiving, from the at least one sensor, data captured by the sensor;
identifying a person based on the data captured by the sensor using a trained machine learning model; and
indicating an identity of the person using at least one of the visual display and the audio output device, wherein the wearable device comprises glasses, and wherein the identity of the person is determined by using at least one of the methods selected from the group consisting of:
visually indicated by displaying a name of the person;
visually indicated using visual augmented reality;
audibly indicated by playing a name of the person; and
visually and audibly indicated using visual and audio augmented reality.

9. A system comprising:
a wearable device comprising a visual display, an audio output device, and at least one sensor;
a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor;
receiving, from the at least one sensor, data captured by the sensor;
identifying a person based on the data captured by the sensor using a trained machine learning model; and
indicating an identity of the person using at least one of the visual display and the audio output device, wherein the wearable device comprises glasses, wherein the sensors comprise at least one of a camera, a microphone, a body odor sensor, and a thermal imaging camera and wherein the data captured by the sensors are used to perform at least one of the methods selected from the group consisting of:
update the trained machine learning model in real-time upon user request and/or command; and
train an un-trained machine learning model in real-time upon user request and/or command.

10. The system of claim 9 , wherein the real-time training update of the machine learning model using the data from a sensor and or multiple sensors can be initiated or triggered using keywords or key phrases.

11. The system of claim 9 , wherein the real-time training of the machine learning model using the data from a sensor and or multiple sensors can be initiated or triggered using keywords or key phrases.

Assignees

• Univ Texas

Inventors

• Stockton Patrick Michael
• John Eugene Britto

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V40/197

  Matching; Classification · CPC title

• G06V40/25

  Recognition of walking or running movements, e.g. gait recognition · CPC title

• G06N20/00

  Machine learning · CPC title