Automated control of microfluidic devices based on machine learning

Patent metadata

Field	Value
Publication number	US-11061042-B2
Application number	US-201916376380-A
Country	US
Kind code	B2
Filing date	Apr 5, 2019
Priority date	Apr 26, 2017
Publication date	Jul 13, 2021
Grant date	Jul 13, 2021

Abstract

A system is provided to automatically monitor and control the operation of a microfluidic device using machine learning technology. The system receives images of a channel of a microfluidic device collected by a camera during operation of the microfluidic device. Upon receiving an image, the system applies a classifier to the image to classify the operation of the microfluidic device as normal, in which no adjustment to the operation is needed, or as abnormal, in which an adjustment to the operation is needed. When an image is classified as normal, the system may make no adjustment to the microfluidic device. If, however, an image is classified as abnormal, the system may output an indication that the operation is abnormal, output an indication of a needed adjustment, or control the microfluidic device to make the needed adjustment.

First claim

We claim:

1. A computing system comprising:
one or more computer-readable storage mediums storing computer-executable instructions that, when executed, control the computing device to:
receive images of a channel of a fluidic device during operation of the fluidic device;
apply a classifier to the images to generate a label to whether the fluidic device is operating normally or abnormally, the classifier having been trained using training data that includes sample images having labels indicating whether the sample images are of normal or abnormal operation, the sample images being of a channel of a training fluidic device taken during operation of the training fluidic device;
identify based on the labels controls for the fluidic device; and
direct operation of the fluidic device based on the identified controls; and

one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums.

2. The computing system of claim 1 wherein the fluidic device includes a micromixer for mixing two or more fluids with a microchannel.

3. The computing system of claim 2 wherein the two or more fluids are reactants.

4. The computing system of claim 2 wherein the two or more fluids include an acid or a base to control pH balance in another fluid.

5. The computing system of claim 2 wherein the two or more fluids include an acid or a base to control pH balance in another fluid.

6. The computing system of claim 1 wherein a control is a change in a flow rate of a fluid.

7. The computing system of claim 1 wherein a control is to discard malformed droplets of a double emulsion.

8. The computing system of claim 1 wherein during initial operation of the fluidic device, the controls direct the fluidic device to operate normally.

9. The computing system of claim 1 wherein the fluidic device includes a microswitch and wherein the controls direct the microswitch to sort cells of a target shape into a target bin and other cells into another bin.

10. The computing system of claim 1 wherein the fluidic device is a microencapsulation device with a microswitch and wherein the controls direct the microswitch to deliver well-formed microcapsules to one bin and malformed microcapsules to another bin.

11. A computing system comprising:
one or more computer-readable storage mediums storing computer-executable instructions that, when executed, control the computing device to:
receive images of a channel of a fluidic device during operation of the fluidic device; and
for each image,
apply a classifier to the image to generate a label for the image, the classifier trained using training data that includes images that are each labeled with an indication of whether the image represents normal or abnormal operation; and
when the label indicates to adjust the operation of the fluidic device, control the fluidic device to adjust the operation of the fluidic device in accordance with the label; and

one or more processors for executing the computer-executable instructions stored in the one or more computer-readable storage mediums.

12. The computing system of claim 11 wherein a label indicates an abnormal operation of the fluidic device.

13. The computing system of claim 12 wherein when a label indicates an abnormal operation, flow rate of a fluid that enters the fluidic device is adjusted.

14. The computing system of claim 11 wherein the fluidic device includes a micromixer for mixing two or more fluids with a microchannel.

15. The computing system of claim 14 wherein the two or more fluids are reactants.

16. The computing system of claim 11 wherein an adjustment is a change in a flow rate of a fluid.

17. The computing system of claim 11 wherein an adjustment is to discard malformed droplets of a double emulsion.

18. The computing system of claim 11 wherein the fluidic device includes a microswitch and wherein the adjustments direct the microswitch to sort cells of a target shape into a target bin and other cells into another bin.

19. The computing system of claim 11 wherein the fluidic device is a microencapsulation device with a microswitch and wherein the adjustments direct the microswitch to deliver well-formed microcapsules to one bin and malformed microcapsules to another bin.

20. The computing system of claim 11 wherein the classifier includes a convolution neural network and a sub-classifier.

21. A method performed by a computing system, the method comprising:
receiving images of a channel of a production fluidic device;
applying an algorithm to the images to determine whether the images represent normal or abnormal operation of the production fluidic device, the algorithm generated using a machine learning technique based on sample images collected during normal operation and abnormal operation of a fluidic device; and
identifying controls for controlling operation of the production fluidic device based on the determination as to whether the images represent normal or abnormal operation of the production fluidic device.

22. The method of claim 21 further comprising controlling the production fluidic device based on the determination.

23. The method of claim 21 wherein the classifier includes a convolution neural network to generate features for the images and a sub-classifier to generate labels for the images based on the generated features of the images.

24. The method of claim 23 wherein the sub-classifier generates the labels for the images further based on an additional feature not generated by the convolution neural network.

25. The method of claim 21 wherein when the determination indicates an abnormal operation, flow rate of a fluid that enters the fluidic device is adjusted.

26. The method of claim 21 wherein the fluidic device includes a micromixer for mixing two or more fluids with a microchannel.

27. The method of claim 26 wherein the two or more fluids are reactants.

28. The method of claim 26 wherein the two or more fluids include an acid or a base to control pH balance in another fluid.

29. The method of claim 21 wherein an identified control is to change in a flow rate of a fluid.

30. The method of claim 21 wherein an identified control is to discard malformed droplets of a double emulsion.

31. The method of claim 21 wherein the fluidic device includes a microswitch and wherein the identified controls are to direct the microswitch to sort cells of a target shape into a target bin and other cells into another bin.

32. The method of claim 21 wherein the fluidic device is a microencapsulation device with a microswitch and wherein the controls direct the microswitch to deliver well-formed microcapsules to one bin and malformed microcapsules to another bin.

33. A computing system comprising:
one or more computer-readable storage mediums storing computer-executable instructions that, when executed, control the computing device to:
receive an image of a channel of a microfluidic device;
apply a classifier to the image to generate a label for the image, the classifier trained using training data that includes images with labels indicating whether the images represent normal or abnormal operation of the microfluidic device;
identify an adjustment for controlling operation of the mi

Assignees

• L Livermore Nat Security Llc

Inventors

• Giera Brian
• Duoss Eric B
• Nguyen Du
• Smith William
• Talathi Sachin Subhash
• Wilson Aaron Creighton
• Ye Congwang

Classifications

• G05D7/0694 Primary

  by action on throttling means or flow sources of very small size, e.g. microfluidics (microvalves F16K99/0001; microstructural devices per se B81B) · CPC title

• G05B2219/37333

  Position of control valve and position of controlled actuator · CPC title

• G01N35/00871 Primary

  Communications between instruments or with remote terminals · CPC title

• B01L3/502784

  specially adapted for droplet or plug flow, e.g. digital microfluidics · CPC title

• G05D7/0635

  by action on throttling means (G05D7/0688, G05D7/0694 take precedence) · CPC title