System and method for model-predictive-control-based micro-assembly control with the aid of a digital computer

Patent metadata

Field	Value
Publication number	US-12307629-B2
Application number	US-202418745811-A
Country	US
Kind code	B2
Filing date	Jun 17, 2024
Priority date	Nov 16, 2020
Publication date	May 20, 2025
Grant date	May 20, 2025

Abstract

System and method that allow to jointly cause movement of multiple micro-and-nano-objects to desired positions are described. A high speed camera tracks the locations of the objects. An array of electrodes is used to generate a dynamic potential energy landscape for manipulating objects with both DEP and EP forces. One or more computing devices are used to: process images captured by the camera to estimate positions of the objects; use model predictive control optimization to obtain trajectories and electrode potentials for moving at least some of the objects from estimated positions to further positions; and control the electrodes based on electrode potentials.

First claim

What is claimed is:

1. A system for model-predictive-control-based micro-assembly control with the aid of a digital computer, comprising:
a plurality of processors configured to execute computer-executable code, the processors comprising at least one of one or more graphics processing units (GPUs) and one or more of tensor processing units (TPUs) and are configured to:
obtain one or more parameters of a system for positioning a plurality of chiplets, each of the chiplets comprising a micro-object, the system comprising a plurality of electrodes, the electrodes configured to induce a movement of the chiplets when the chiplets are suspended in a fluid proximate to the electrodes upon a generation of one or more electric potentials by one or more of the electrodes;
model capacitance between the chiplets and the electrodes and the electrodes and capacitance between the chiplets based on the parameters of the system;
estimate positions of the chiplets based on images taken by at least one camera;
receive further positions of at least some of the chiplets;
perform model predictive control (MPC) optimization to derive based on the capacitance modeling a control scheme for moving the at least some chiplets from the positions to the further positions along trajectories and the electrode potentials necessary for the at least some chiplets to travel the along trajectories, wherein the trajectories are parametrized as smooth, time-dependent functions during the MPC optimization and the electrode potentials are parametrized as smooth time-and-space-dependent functions during the MPC optimization, wherein the processors are further configured to impose one or more constraints when performing the MPC optimization, one of the constraints comprising a minimum distance between the trajectories of the chiplets when performing the MPC optimization; and
control the electrodes to generate the electrode potentials in the control scheme.

2. A system according to claim 1 , wherein the processors comprise one or more of the GPUs and one or more of the TPUs.

3. A system according to claim 2 , wherein two or more of the processors work in parallel.

4. A system according to claim 1 , wherein at least one of one or more of the GPUs and one or more of the TPUs utilizes JAX Python framework when performing the MPC optimization.

5. A system according to claim 1 , wherein the constraints further comprise a constraint on a magnitude of the electrode potentials and a constraint on how far the chiplets can move.

6. A system according to claim 1 , the processors further configured to set one or more of the constraints based on user input.

7. A system according to claim 1 , wherein a second order information of a loss function is used during the MPC optimization.

8. A system according to claim 1 , the processors further configured to map the electrode potentials to a plurality of further images, wherein the further images are used to control the electrodes.

9. A system according to claim 8 , the processors further configured to use a video projector to project the images to photo-transistors in control of the electrodes.

10. A method for model-predictive-control-based micro-assembly control with the aid of a digital computer micro-assembly control with the aid of a digital computer, comprising:
obtaining by one or more of a plurality of processors configured to execute computer-executable code, the processors comprising at least one of one or more graphics processing units (GPUs) and one or more of tensor processing units (TPUs), the one or more parameters of a system for positioning a plurality of chiplets, each of the chiplets comprising a micro-object, the system comprising a plurality of electrodes, the electrodes configured to induce a movement of the chiplets when the chiplets are suspended in a fluid proximate to the electrodes upon a generation of one or more electric potentials by one or more of the electrodes;
modeling by one or more of the plurality of processors capacitance between the chiplets and the electrodes and the electrodes and capacitance between the chiplets based on the parameters of the system;
estimating by one or more of the plurality of processors positions of the chiplets based on images taken by at least one camera;
receiving by one or more of the plurality of processors further positions of at least some of the chiplets;
performing model predictive control (MPC) optimization to derive based on the capacitance modeling a control scheme for moving the at least some chiplets from the positions to the further positions along trajectories and the electrode potentials necessary for the at least some chiplets to travel the along trajectories, wherein the trajectories are parametrized as smooth, time-dependent functions during the MPC optimization and the electrode potentials are parametrized as smooth time-and-space-dependent functions during the MPC optimization, wherein a second order information of a loss function is used during the MPC optimization; and
controlling the electrodes to generate the electrode potentials in the control scheme.

11. A method according to claim 10 , wherein the processors comprise one or more of the GPUs and one or more of the TPUs.

12. A method according to claim 11 , wherein two or more of the processors work in parallel.

13. A method according to claim 10 , wherein at least one of one or more of the GPUs and one or more of the TPUs utilizes JAX Python framework when performing the MPC optimization.

14. A method according to claim 10 , further comprising imposing one or more constraints when performing the MPC optimization, one of the constraints comprising a minimum distance between the trajectories of the chiplets when performing the MPC optimization.

15. A method according to claim 14 , wherein the constraints further comprise a constraint on a magnitude of the electrode potentials and a constraint on how far the chiplets can move.

16. A method according to claim 14 , further comprising setting one or more of the constraints based on user input.

17. A method according to claim 10 , further comprising mapping the electrode potentials to a plurality of further images, wherein the further images are used to control the electrodes.

18. A method according to claim 17 , further comprising using a video projector to project the images to photo-transistors in control of the electrodes.

19. A system for model-predictive-control-based micro-assembly control with the aid of a digital computer, comprising:
a plurality of processors configured to execute computer-executable code, the processors comprising at least one of one or more graphics processing units (GPUs) and one or more of tensor processing units (TPUs) and are configured to:
obtain one or more parameters of a system for positioning a plurality of chiplets, each of the chiplets comprising a micro-object, the system comprising a plurality of electrodes, the electrodes configured to induce a movement of the chiplets when the chiplets are suspended in a fluid proximate to the electrodes upon a generation of one or more electric potentials by one or more of the electrodes;
model capacitance between the chiplets and the electrodes and the electrodes and capacitance between the chiplets based on the parameters of the system;
estimate positions of the chiplets based on images taken by at least one camera;
receive further positions of at least some of the chiplets;
perform model predictive control (MPC) optimization to derive based on the capacitance modeling a control

Assignees

• Xerox Corp

Inventors

• Matei Ion
• De Kleer Johan

Classifications

• H10W72/07183

  Means for monitoring · CPC title

• H10W72/0711

  Apparatus therefor · CPC title

• B82B3/0076 Primary

  Methods for manipulating nanostructures not provided for in groups B82B3/0066 - B82B3/0071 · CPC title

• B81C99/002

  Apparatus for assembling MEMS, e.g. micromanipulators (micromanipulators per se B25J7/00) · CPC title

• G06T15/005

  General purpose rendering architectures · CPC title