System and method for machine-learning enabled micro-assembly control with the aid of a digital computer

Patent metadata

Field	Value
Publication number	US-11893327-B2
Application number	US-202017121411-A
Country	US
Kind code	B2
Filing date	Dec 14, 2020
Priority date	Dec 14, 2020
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

System and method that allow utilize machine learning algorithms to move a micro-object to a desired position are described. A sensor such as a high speed camera or capacitive sensing, tracks the locations of the objects. A dynamic potential energy landscape for manipulating objects is generated by controlling each of the electrodes in an array of electrodes. One or more computing devices are used to: estimate an initial position of a micro-object using the sensor; generate a continuous representation of a dynamic model for movement of the micro-object due to electrode potentials generated by at least some of the electrodes and use automatic differentiation and Gauss quadrature rules on the dynamic model to derive optimum potentials to be generated by the electrodes to move the micro-object to the desired position; and map the calculated optimized electrode potentials to the array to activate the electrodes.

First claim

What is claimed is:

1. A system for machine-learning enabled micro-assembly control with an aid of a digital computer, comprising:
one or more processors implementing one or more machine learning algorithms, the one or more processors configured to:
obtain one or more parameters of a system for positioning a micro-object, the system comprising a plurality of electrodes, each of the electrodes being programmable, the electrodes configured to induce a movement of the micro-object when the micro-object is proximate to the electrodes upon a generation of one or more electric potentials by one or more of the electrodes;
model capacitance between the micro-object and the electrodes using the parameters;
estimate a position of the micro-object with a sensor;
receive further position of the micro-object;
define using the capacitance a continuous representation of a dynamic model for movement of the micro-object due to electrode potentials generated by at least one of the plurality of the electrodes, wherein the dynamic model is associated with constraints expressed as expectations;
apply one of the machine learning algorithms to at least a portion of the continuous representation of the dynamic model to perform an optimization of electrode potentials to be generated by at least one of the plurality of the electrodes to move the micro-object from the estimated position to the further position; and
use the optimized electrode potentials to actuate at least one of the plurality of the electrodes to induce the movement of the micro-object from the estimated position to the further position.

2. A system according to claim 1 , wherein one of the constraints comprises a potential energy of the micro-object and the potential energy is approximated during the optimization using Gauss quadrature rules.

3. A system according to claim 2 , the one or more processors further configured to:
represent a function that determines an allocation of the optimized electrode potentials as a neural network associated with weights and biases.

4. A system according to claim 3 , wherein the one machine learning algorithm is automatic differentiation, the one or more processors further configured to:
use the automatic differentiation to compute partial derivatives of the potential energy as a function of the weights and biases associated with the neural network.

5. A system according to claim 3 , wherein the neural network comprises two hidden layer and uses tanh as an activation function.

6. A system according to claim 1 , wherein the further position is separated by a distance in a direction from the estimated position, the one or more processors further configured to:
receive an additional position that is separated by the distance from the estimated position in a direction opposite to the direction in which the further position is separated from the estimated position; and
change a sign of the function to optimize the potentials for moving the micro-object from the estimated position to the additional position.

7. A system according to claim 1 , wherein the movement from the estimated position to the further position is in one dimension, the one or more processors further configured to:
receive an additional position that is separated from the estimated position in two dimensions; and
apply a rotational transformation to the optimized electrode potentials to derive the electrode potentials to be generated by the electrodes for moving the micro-object from the estimated position to the additional position.

8. A system according to claim 1 , the one or more processors further configured to:
perform a plurality of simulations of the capacitance between the electrodes and the micro-object; and
define a function describing the capacitance between the micro-object and each of the electrodes as a function of a distance between the micro-object and that electrode.

9. A system accordingto claim 1 , one or more of the processors comprise one or more of at least one central processingunit (CPU) and at least one graphics processing unit (GPU).

10. A system according to claim 1 , wherein at least one of the plurality of the electrodes is rectangular.

11. A method for machine-learning enabled micro-assembly control with an aid of a digital computer, comprising:
obtaining, by one or more processors implementing one or more machine learning algorithms, one or more parameters of a system for positioning a micro-object, the system comprising a plurality of programmable electrodes, the electrodes configured to induce a movement of the micro-objects when the micro-objects are proximate to the electrodes upon a generation of one or more electric potentials by one or more of the electrodes;
modeling by one or more of the processors capacitance between the micro-object and the electrodes using the parameters;
estimating by one or more of the processors a position of the micro-object based on at least one sensor measurement;
receive by one or more of the processors further position of the micro-object;
define by one or more of the processors using the capacitance a continuous representation of a dynamic model for movement of the micro-object due to electrode potentials generated by at least one of the plurality of the programmable electrodes, wherein the dynamic model is associated with constraints expressed as expectations;
apply by one or more of the processors one of the machine learning algorithms to at least a portion of the continuous representation of the dynamic model to perform an optimization of electrode potentials to be generated by at least one of the plurality of the programmable electrodes to move the micro-object from the estimated position to the further position; and
using the optimized electrode potentials to actuate at least one of the plurality of the programmable electrodes to induce the movement of the micro-object from the estimated position to the further position.

12. A method according to claim 11 , wherein one of the constraints comprises a potential energy of the micro-object and the potential energy is approximated during the optimization using Gauss quadrature rules.

13. A method according to claim 12 , further comprising:
representing a function that determines an allocation of the optimized electrode potentials as a neural network associated with weights and biases.

14. A method according to claim 13 , wherein the one machine learning algorithm is automatic differentiation, further comprising:
using the automatic differentiation to compute partial derivatives of the potential energy as a function of the weights and biases associated with the neural network.

15. A method according to claim 13 , wherein the neural network comprises two hidden layers and uses tanh as an activation function.

16. A method according to claim 11 , wherein the further position is separated by a distance in a direction from the estimated position, further comprising:
receiving an additional position that is separated by the distance from the estimated position in a direction opposite to the direction in which the further position is separated from the estimated position; and
changing a sign of the function to optimize the potentials for moving the micro-object from the micro-object from the estimated position to the additional position.

17. A method according to claim 11 , wherein the movement from the estimated position to the further position is in one dimension, the one or more processors further configured to:
receiving an additional position that is separated from the estimated positio

Assignees

• Xerox Corp

Inventors

• Matei Ion
• Plochowietz Anne
• Nelaturi Saigopal
• De Kleer Johan
• Lu Jeng Ping
• Crawford Lara S
• Chow Eugene M

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F30/27 Primary

  using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model · CPC title

• G06F30/17

  Mechanical parametric or variational design · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title