Eyeglasses having inward-facing and outward-facing thermal cameras

Patent metadata

Field	Value
Publication number	US-10151636-B2
Application number	US-201715833079-A
Country	US
Kind code	B2
Filing date	Dec 6, 2017
Priority date	Jun 14, 2015
Publication date	Dec 11, 2018
Grant date	Dec 11, 2018

Abstract

Described herein are systems and methods for detecting a physiological response based on thermal measurements while accounting for effects of the environment. In one embodiment, a system includes an inward-facing head-mounted thermal camera (CAM in ) that takes thermal measurements of a region of interest (TH ROI ) on a user's face, and an outward-facing head-mounted thermal camera (CAM out ) that takes thermal measurements of the environment (TH ENV ). CAM in does not occlude the region of interest, and the system further includes a computer that detects the physiological response based on TH ROI and TH ENV . Optionally, the computer generates feature values based on sets of TH ROI and TH ENV , and utilizes a machine learning-based model to detect, based on the feature values, the physiological response.

First claim

We claim:

1. A system configured to detect a physiological response, comprising:
an inward-facing head-mounted thermal camera (CAM in ) configured to take thermal measurements of a region of interest (TH ROI ) on a user's face; wherein CAM in does not occlude the region of interest (ROI) and is located less than 15 cm from the user's face;
an outward-facing head-mounted thermal camera (CAM out ) configured to take thermal measurements of the environment (TH ENV ), and is located less than 15 cm from the face; and
a computer configured to detect the physiological response from feature values generated based on TH ROI and TH ENV .

2. The system of claim 1 , wherein the computer is further configured to generate the feature values based on sets of TH ROI and TH ENV , and to utilize a machine learning-based model to detect, based on the feature values, the physiological response; wherein responsive to receiving a first set of measurements in which TH ROI reaches a first threshold while TH ENV does not reach a second threshold, the computer detects the physiological response; and wherein responsive to receiving a second set of measurements in which TH ROI reaches the first threshold while TH ENV reaches the second threshold, the computer does not detect the physiological response.

3. The system of claim 1 , wherein the field of view (FOV) of CAM in is larger than the FOV of CAM out , and the noise equivalent differential temperature (NEDT) of CAM in is lower than NEDT of CAM out .

4. The system of claim 1 , wherein, on average, detections of the physiological responses based on TH ROI and TH ENV are more accurate compared to detections of the physiological responses based on TH ROI without TH ENV ; and wherein CAM in has a field of view (FOV) smaller than 80°, CAM out has a FOV larger than 80°, and CAM in has more sensing elements than CAM out .

5. The system of claim 1 , wherein the angle between the optical axes of CAM in and CAM out is at least one of the following angles: 45°, 90°, 130°, 170°, and 180°; and wherein CAM in and CAM out are based on sensors of the same type and are located less than 5 cm apart.

6. The system of claim 1 , further comprising an eyeglasses frame configured to be worn on the user's head; wherein CAM in and CAM out each weigh below 5 g and are physically coupled to the frame.

7. The system of claim 1 , wherein the region of interest includes a region on the forehead of the user and the physiological response comprises at least one of the following: stress, a headache, and a stroke.

8. The system of claim 1 , wherein the region of interest includes a region on the nose of the user and the physiological response is an allergic reaction.

9. The system of claim 1 , further comprising a second outward-facing head-mounted thermal camera (CAM out2 ) configured to take thermal measurements of the environment (TH ENV2 ); wherein there is an angle of at least 30° between the optical axes of CAM out and CAM out2 ; responsive to receiving a first set of measurements in which TH Rot reach a first threshold while the difference between TH ENV and TH ENV2 does not reach a second threshold, the computer detects the physiological response; and responsive to receiving a second set of measurements in which TH ROI reach the first threshold while the difference between TH ENV and TH ENV2 reaches the second threshold, the computer does not detect the physiological response.

10. The system of claim 1 , further comprising a second outward-facing head-mounted thermal camera (CAM out2 ) configured to take thermal measurements of the environment (TH ENV2 ); wherein there is an angle of at least 30° between the optical axes of CAM out and CAM out2 ; and wherein the computer is configured to detect the physiological response based on a difference between TH ROI , TH ENV , and TH ENV2 , while taking into account the angle between the optical axes of CAM out and CAM out2 and a graph of responsivity as function of the angle from the optical axes of each of CAM out and CAM out2 .

11. The system of claim 1 , wherein CAM in and CAM out are located to the right of the vertical symmetry axis that divides the user's face, and the ROI is on the right side of the face; and further comprising a second inward-facing head-mounted thermal camera (CAM in2 ) and a second outward-facing head-mounted thermal camera (CAM out2 ) located to the left of the vertical symmetry axis; CAM in2 is configured to take thermal measurements of a second ROI (TH ROI2 ) on the left side of the face, and does not occlude ROI 2 ; CAM out2 is configured to take thermal measurements of an environment (TH ENV2 ) that is more to the left relative to the environment measured by CAM out ; wherein the computer is configured to detect the physiological response also based on TH ROI2 and TH ENV2 .

12. The system of claim 1 , wherein the optical axes of CAM in and CAM out are above the Frankfort horizontal plane; and further comprising a second inward-facing head-mounted thermal camera (CAM in2 ) and a second outward-facing head-mounted thermal camera (CAM out2 ), located such that their optical axes are below the Frankfort horizontal plane, and configured to take thermal measurements TH ROI2 and TH ENV2 , respectively; wherein the computer is configured to detect the physiological response also based on TH ROI2 and TH ENV2 .

13. The system of claim 1 , further comprising a sensor configured to take measurements (m conf ) that are indicative of at least one of the following: an extent of the user's activity, an orientation of the user's head, and a change in a position of the user's body; wherein the sensor comprises one or more of the following sensors: (i) a movement sensor that is physically coupled to a frame worn on the user's head or to a wearable device worn by the user, (ii) a visible-light camera configured to take images of the user, and (iii) an active 3D tracking device configured to emit electromagnetic waves and generate 3D images based on received reflections of the emitted electromagnetic waves; and wherein the computer is further configured to detect the physiological response also based on m conf .

14. The system of claim 13 , wherein the computer is further configured to generate feature values based on TH ROI , TH ENV , and m conf , and to utilize a model to detect the physiological response based on the feature values; wherein the model was trained based on: a first set of previous TH ROI , TH ENV , and m conf taken while the user was walking or running, and a second set of previous TH ROI , TH ENV , and m conf taken while the user was sitting or standing.

15. The system of claim 13 , wherein m conf are indicative of bending the head down above a certain degree from the normal to earth; wherein the computer is further configured to generate feature values based on TH ROI , TH ENV , and m conf , and to utilize a model to detect the physiological response based on the feature values; wherein the model was trained based on: a first set of TH ROI , TH ENV , and m conf taken while the user was bending the head down above the certain degree, and a second set of previous TH ROI , TH ENV , and m conf taken while the user was not bending the head down above the certain degree.

16. The system of claim 1 , further comprising an anemometer that is physically coupled to a frame worn on the user's head, is located less than 15 cm from the face, and is configured to provide a value indicative of a speed of air directed at the face (m wind ); wherein the computer is further configured to detect the physiological respons

Assignees

• Facense Ltd

Inventors

• Tzvieli Arie
• Thieberger Gil
• Frank Ari M

Classifications

• A61B5/015 Primary

  By temperature mapping of body part · CPC title

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title

• G16H50/20

  for computer-aided diagnosis, e.g. based on medical expert systems · CPC title

• A61B2562/0271

  Thermal or temperature sensors · CPC title

• G01J5/0025

  Living bodies (ear thermometers G01J5/0011; detecting, measuring or recording for diagnostic purposes A61B5/00) · CPC title