Camera-based droplet guidance and detection

Patent metadata

Field	Value
Publication number	US-2024058168-A1
Application number	US-202318236375-A
Country	US
Kind code	A1
Filing date	Aug 21, 2023
Priority date	Aug 19, 2022
Publication date	Feb 22, 2024
Grant date	—

Abstract

Systems and methods are provided for determining a likelihood that a drop released from a bottle will land in an eye of a user. Each of an eye of a user and a tip of a bottle and illuminated and imaged to provide at least one image. An orientation of the bottle is determined via an inertial measurement unit, and a likelihood that a drop released from the bottle will land in the eye of the user is determined from the determined orientation and the at least one image.

First claim

1 . A method comprising:
illuminating each of an eye of a user and a tip of a bottle; imaging each of the eye of the user and the tip of the bottle with a camera to provide at least one image; determining an orientation of the bottle via an inertial measurement unit; and determining a likelihood that a drop released from the bottle will land in the eye of the user from the determined orientation of the bottle and the at least one image.

2 . The method of claim 1 , further comprising detecting, from the at least one image, a release of a drop from the bottle, and determining the likelihood that the drop released from the bottle is performed in response to detecting the release of the drop from the bottle.

3 . The method of claim 2 , further comprising storing a time at which the drop was released and the determined likelihood that the drop released will land in the eye of the user at a non-transitory computer readable medium.

4 . The method of claim 1 , wherein illuminating each of eye of the user and the tip of the bottle comprises illuminating each of eye of the user and the tip of the bottle with light of a specific wavelength, and imaging each of the eye of the user and the tip of the bottle comprises imaging each of the eye of the user and the tip of the bottle through a spectral filter that attenuates light outside of a band of wavelengths including the specific wavelength.

5 . The method of claim 1 , wherein illuminating each of eye of the user and the tip of the bottle comprises modulating an illumination source to pulse in synchrony with a frame acquisition rate of the camera to produce a plurality of illuminated images and a plurality of non-illuminated images, and determining a likelihood that the drop released from the bottle will land in the eye of the user comprises subtracting an unilluminated image from each illuminated image to provide a background subtracted image.

6 . The method of claim 1 , wherein illuminating each of eye of the user and the tip of the bottle comprises illuminating the eye of the user with a first illumination source and illuminating the tip of the bottle with a second illumination source, the first illumination source being inactive at least a portion of a time for which the second illumination source is active.

7 . The method of claim 1 , wherein determining the likelihood that the drop released from the bottle will land in the eye of the user comprises providing the at least one image and the determined orientation to a machine learning model.

8 . The method of claim 1 , wherein determining the likelihood that the drop released from the bottle will land in the eye of the user comprises determining a relative location of the eye of the user and the tip of the bottle from the at least one image.

9 . The method of claim 1 , further comprising providing one of an audible, visible, or tactile signal to the user representing the likelihood that the drop released from the bottle will land in the eye of the user.

10 . The method of claim 1 , further comprising trigging a release of a drop from the bottle via an automated mechanism when the likelihood that the drop released from the bottle will land in the eye of the user exceeds a threshold value.

11 . A system comprising:
an illumination source positioned to illuminate each of an eye of a user and a tip of a bottle when the when the device and the bottle with which it is associated is in an appropriate position to deliver mediation to the eye; a camera that images each of the eye of the user and the tip of the bottle while the illumination source is active to provide at least one image; an inertial measurement unit that determines an orientation of the bottle; and an alignment component that determines a likelihood that a drop released from the bottle will land in the eye of the user from the determined orientation of the bottle and the at least one image.

12 . The system of claim 11 , further comprising a housing configured to engage with the bottle, the housing containing at least a portion of the alignment component.

13 . The system of claim 11 , further comprising an output device that provides one of an audible, visible, or tactile signal to the user representing the likelihood that the drop released from the bottle will land in the eye of the user.

14 . The system of claim 11 , further comprising trigging a release of a drop from the bottle via an automated mechanism when the likelihood that the drop released from the bottle will land in the eye of the user exceeds a threshold value.

15 . The system of claim 11 , further comprising a droplet detection component that processes images received from the camera to determine if the drop has been released from the tip of the bottle.

16 . A method for monitoring compliance in the application of eye drops for a user, the method comprising:
illuminating each of an eye of a user and a tip of a bottle; imaging each of the eye of the user and the tip of the bottle with a camera to provide at least one image; detecting, from the at least one image, a release of a drop from the bottle; determining an orientation of the bottle via an inertial measurement unit; and determining a likelihood that a drop released from the bottle will land in the eye of the user from the determined orientation of the bottle and the at least one image in response to detecting the release of the drop from the bottle.

17 . The method of claim 16 , wherein illuminating each of eye of the user and the tip of the bottle comprises illuminating each of eye of the user and the tip of the bottle with light of a specific wavelength, and imaging each of the eye of the user and the tip of the bottle comprises imaging each of the eye of the user and the tip of the bottle through a spectral filter that attenuates light outside of a band of wavelengths including the specific wavelength.

18 . The method of claim 16 , wherein illuminating each of eye of the user and the tip of the bottle comprises modulating an illumination source to pulse in synchrony with a frame acquisition rate of the camera to produce a plurality of illuminated images and a plurality of non-illuminated images, and determining a likelihood that the drop released from the bottle will land in the eye of the user comprises subtracting an unilluminated image from each illuminated image to provide a background subtracted image.

19 . The method of claim 16 , wherein illuminating each of eye of the user and the tip of the bottle comprises illuminating the eye of the user with a first illumination source and illuminating the tip of the bottle with a second illumination source, the first illumination source being inactive at least a portion of a time for which the second illumination source is active.

20 . The method of claim 16 , further comprising storing a time at which the drop was released and the determined likelihood that the drop released will land in the eye of the user at a non-transitory computer readable medium.

Assignees

• Twenty Twenty Therapeutics Llc

Inventors

• Rumyantsev Oleg
• Sinha Supriyo
• Azar Dimitri
• Luo Kevin
• Eyal Michael

Classifications

• A61F9/0026 Primary

  Ophthalmic product dispenser attachments to facilitate positioning near the eye · CPC title

• A61B34/76

  Manipulators having means for providing feel, e.g. force or tactile feedback · CPC title

• A61B2034/2057

  Details of tracking cameras · CPC title

• A61F9/0008 Primary

  Introducing ophthalmic products into the ocular cavity or retaining products therein (putting in contact lenses A61F9/0061; introducing or retaining media in cavities of the body in general A61M31/00) · CPC title

• G16H20/13

  delivered from dispensers · CPC title