Robotic vision system with variable lens for value chain networks

Patent metadata

Field	Value
Publication number	US-12511630-B2
Application number	US-202217683215-A
Country	US
Kind code	B2
Filing date	Feb 28, 2022
Priority date	Dec 18, 2020
Publication date	Dec 30, 2025
Grant date	Dec 30, 2025

Abstract

A dynamic vision system includes a variable focus liquid lens optical assembly. The dynamic vision system includes a variable lighting assembly. The dynamic vision system includes a control system configured to adjust one or more optical parameters and data collected from the variable focus liquid lens optical assembly in real time. The dynamic vision system includes a control system configured to adjust the variable lighting assembly. The dynamic vision system includes a processing system that dynamically learns on a training set of outcomes, parameters, and data collected from the variable focus liquid lens optical assembly to train a set of machine learning models to control the variable focus liquid lens optical assembly to optimize collection of data for processing by the set of machine learning models.

First claim

The invention claimed is:

1 . A robotic vision system comprising:
an optical assembly including at least one sensor, a variable focus liquid lens, and a photon capture board; a digital twin system configured to:
generate a digital twin of the optical assembly;
adjust at least one parameter of a plurality of digital twin optical parameters associated with the digital twin of the optical assembly;
execute a simulation based on the plurality of digital twin optical parameters as adjusted; and
collect data from the simulation; and
a processing system configured to dynamically learn on a training set to train an artificial intelligence model to recognize an object, wherein the training set includes outcomes, optical parameters, and data collected from the optical assembly and the data collected from the simulation, wherein the artificial intelligence model is trained for at least one of: classification, prediction, or optimization related decisions about the object, and wherein the artificial intelligence model builds a three-dimensional representation of the object in a single step without an intermediate step of processing into two-dimensional images.

2 . The robotic vision system of claim 1 further comprising a control system configured to adjust one the optical parameters in real time.

3 . The robotic vision system of claim 1 wherein the artificial intelligence model determines at least one of a position, an orientation, or a motion of the object.

4 . The robotic vision system of claim 1 wherein the artificial intelligence model is a neural network.

5 . The robotic vision system of claim 1 wherein the at least one sensor includes at least one of cameras, LIDARs, RADARs, SONARs, thermal imaging sensor, hyperspectral imaging sensor, illuminance sensors, force sensors, torque sensors, velocity sensors, acceleration sensors, position sensors, proximity sensors, gyro sensors, sound sensors, motion sensors, location sensors, load sensors, temperature sensors, touch sensors, depth sensors, ultrasonic range sensors, infrared sensors, chemical sensors, magnetic sensors, inertial sensors, gas sensors, humidity sensors, pressure sensors, viscosity sensors, flow sensors, object sensors, or tactile sensors.

6 . The robotic vision system of claim 5 wherein the processing system is configured to temporally combine output from two or more sensors using conditional probabilities to create a combined view of the object that is richer and includes information about at least one of: a position, an orientation, or a motion of the object.

7 . The robotic vision system of claim 1 wherein the processing system generates a recommendation for a fluid type to be used in the variable focus liquid lens based on the simulation executed by the digital twin system.

8 . The robotic vision system of claim 1 wherein the processing system identifies suitable imaging components for the optical assembly based on the simulation executed by the digital twin system.

9 . The robotic vision system of claim 8 wherein the digital twin system provides a hypothetical simulation of at least one component of the optical assembly during a design phase before the at least one component is constructed.

10 . The robotic vision system of claim 1 wherein the artificial intelligence model automatically predicts hypothetical situations for simulation with the digital twin system.

11 . A method comprising:
generating a digital twin of an optical assembly, the optical assembly including at least one sensor, a variable focus liquid lens, and a photon capture board; adjusting at least one parameter of a plurality of digital twin optical parameters associated with the digital twin of the optical assembly; executing a simulation based on the plurality of digital twin optical parameters as adjusted; collecting data from the simulation; training an artificial intelligence model on a training set to recognize an object, wherein:
the training set includes outcomes, optical parameters, and data collected from the optical assembly and the data collected from the simulation, and
the artificial intelligence model is trained for at least one of: classification, prediction, or optimization related decisions about the object; and
the artificial intelligence model building a three-dimensional representation of the object in a single step without an intermediate step of processing into two-dimensional images.

12 . The method of claim 11 further comprising adjusting the optical parameters in real time.

13 . The method of claim 11 further comprising determining at least one of: a position, an orientation, or a motion of the object using the artificial intelligence model.

14 . The method of claim 11 wherein the artificial intelligence model is a neural network.

15 . The method of claim 11 wherein the at least one sensor includes at least one of cameras, LIDARs, RADARs, SONARs, thermal imaging sensor, hyperspectral imaging sensor, illuminance sensors, force sensors, torque sensors, velocity sensors, acceleration sensors, position sensors, proximity sensors, gyro sensors, sound sensors, motion sensors, location sensors, load sensors, temperature sensors, touch sensors, depth sensors, ultrasonic range sensors, infrared sensors, chemical sensors, magnetic sensors, inertial sensors, gas sensors, humidity sensors, pressure sensors, viscosity sensors, flow sensors, object sensors, and tactile sensors.

16 . The method of claim 15 further comprising temporally combining output from two or more sensors using conditional probabilities to create a combined view of the object that includes information about at least one of: a position, an orientation, or a motion of the object.

17 . The method of claim 11 further comprising generating a recommendation for a fluid type to be used in the variable focus liquid lens based on the simulation.

18 . The method of claim 11 further comprising identifying suitable imaging components for the optical assembly based on the simulation.

19 . The method of claim 18 further comprising providing a hypothetical simulation of at least one component of the optical assembly during a design phase before the at least one component is constructed.

20 . The method of claim 11 further comprising, using the artificial intelligence model, automatically predicting hypothetical situations for simulation.

Assignees

• Strong Force Vcn Portfolio 2019 Llc

Inventors

• Marinkovich Sava
• Cella Charles H
• Bliven Brent
• Dobrowitsky Joshua
• Cardno Andrew
• Sharma Kunal
• Kell Brad

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N5/025

  Extracting rules from data · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title