Systems and methods for predicting semantics of a particle using semantic segmentation

Patent metadata

Field	Value
Publication number	US-2020167546-A1
Application number	US-201816202707-A
Country	US
Kind code	A1
Filing date	Nov 28, 2018
Priority date	Nov 28, 2018
Publication date	May 28, 2020
Grant date	—

Abstract

System, methods, and other embodiments described herein relate to classifying semantics of a particle or other material component. In one embodiment, a method includes, in response to receiving a particle image, analyzing the particle image to identify characteristics of the particle represented in respective pixels of the particle image to produce a segmented image that groups the pixels into subregions. The method includes identifying semantics of the particle according to at least boundaries between the subregions. The semantics define expected behaviors of the particle in relation to material physics. The method includes providing the segmented image including the semantics as an electronic output.

First claim

What is claimed is:

1 . A semantics system for classifying semantics of a particle, comprising:
one or more processors; a memory communicably coupled to the one or more processors and storing: a segmentation module including instructions that when executed by the one or more processors cause the one or more processors to analyze a particle image to identify characteristics of the particle represented in respective pixels of the particle image and to produce a segmented image that groups the pixels into subregions; and a prediction module including instructions that when executed by the one or more processors cause the one or more processors to identify semantics of the particle according to at least boundaries between the subregions, wherein the semantics define expected behaviors of the particle in relation to material physics, and wherein the prediction module includes instructions to provide the segmented image including the semantics as an electronic output.

2 . The semantics system of claim 1 , wherein the segmentation module includes instructions to analyze the particle image including instructions to use a machine learning algorithm that is a convolutional neural network (CNN) to perform semantic segmentation over the particle image, and
wherein respective ones of the subregions are formed from associated ones of the pixels corresponding to locations on the particle where the characteristics are at least similar, and wherein the characteristics include at least crystallographic parameters, chemical identity, electronic structure, material density, and electrochemical properties.

3 . The semantics system of claim 1 , wherein the pixels of the particle image separately include diffraction patterns indicative of the characteristics of the particle including a physical structure of the particle at a corresponding location, and
wherein the particle image includes diffraction patterns that are patterns of electrons as scattered onto a detector in a transmission electron microscope (TEM) resulting from the electrons interacting with the particle at a location corresponding with a respective one of the pixels.

4 . The semantics system of claim 3 , wherein the segmentation module includes instructions to analyze the particle image including instructions to use a machine learning algorithm that convolves a filter over the diffraction patterns to identify which of the diffraction patterns correlate with ones of the characteristics, and
wherein the prediction module includes instructions to identify the semantics of the particle including instructions to identify relationships between the subregions that define the semantics for the particle.

5 . The semantics system of claim 1 , wherein the prediction module includes instructions to identify the semantics including instructions to map properties of the boundaries between the subregions according to the characteristics of the particle within abutting ones of the subregions.

6 . The semantics system of claim 1 , wherein the prediction module includes instructions to identify the semantics including instructions to map the boundaries including instructions to dynamically control a transmission electron microscope to refine the particle image along the boundaries by re-imaging at least respective ones of the pixels that span the boundaries according to an adjusted scan grid that avoids pixels overlapping the boundaries.

7 . The semantics system of claim 1 , wherein the segmentation module includes instructions to train, using a set of training images depicting different particles and labeled according to associated segments and training semantics, a machine learning algorithm to segment the training images into the subregions and to identify the semantics according to boundaries between the subregions.

8 . The semantics system of claim 1 , wherein the segmentation module includes instructions to receive the particle image including instructions to receive the particle image from a transmission electron microscope (TEM) that scans the particle to produce the particle image.

9 . A non-transitory computer-readable medium for classifying semantics of a particle and including instructions that when executed by one or more processors cause the one or more processors to:
analyze a particle image to identify characteristics of the particle represented in respective pixels of the particle image and to produce a segmented image that groups the pixels into subregions; identify semantics of the particle according to at least boundaries between the subregions, wherein the semantics define expected behaviors of the particle in relation to material physics; and provide the segmented image including the semantics as an electronic output.

10 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to analyze the particle image include instructions to use a machine learning algorithm that is a convolutional neural network (CNN) to perform semantic segmentation over the particle image, and
wherein respective ones of the subregions are formed from associated ones of the pixels corresponding to locations on the particle where the characteristics are at least similar, and wherein the characteristics include at least crystallographic parameters, chemical identity, electronic structure, material density, and electrochemical properties.

11 . The non-transitory computer-readable medium of claim 9 , wherein the pixels of the particle image separately include diffraction patterns indicative of the characteristics of the particle including a physical structure of the particle at a corresponding location, and
wherein the particle image includes diffraction patterns that are patterns of electrons as scattered onto a detector in a transmission electron microscope (TEM) resulting from the electrons interacting with the particle at a location corresponding with a respective one of the pixels.

12 . The non-transitory computer-readable medium of claim 11 , wherein the instructions to analyze the particle image including instructions to use a machine learning algorithm that convolves a filter over the diffraction patterns to identify which of the diffraction patterns correlate with ones of the characteristics, and
wherein the instructions to identify the semantics of the particle include instructions to identify relationships between the subregions that define the semantics for the particle.

13 . The non-transitory computer-readable medium of claim 9 , wherein the instructions to identify the semantics include instructions to map properties of the boundaries between the subregions according to the characteristics of the particle within abutting ones of the subregions.

14 . A method for classifying semantics of a particle, comprising:
in response to receiving a particle image, analyzing the particle image to identify characteristics of the particle represented in respective pixels of the particle image to produce a segmented image that groups the pixels into subregions; identifying semantics of the particle according to at least boundaries between the subregions, wherein the semantics define expected behaviors of the particle in relation to material physics; and providing the segmented image including the semantics as an electronic output.

15 . The method of claim 14 , wherein analyzing the particle image includes using a machine learning algorithm that is a convolutional neural network (CNN) to perform semantic segmentation over the particle image, and
wherein respective ones of the subregions include associated ones of the pixels correspo

Assignees

• Toyota Res Inst Inc

Inventors

• Herring Patrick K

Classifications

• G16H30/20

  for handling medical images, e.g. DICOM, HL7 or PACS · CPC title

• G06K9/00147 Primary

  Physics · mapped topic

• G06N3/08

  Learning methods · CPC title

• G06K9/34

  Physics · mapped topic

• G06K9/726

  Physics · mapped topic