Object-centric and relation-centric graph neural networks for physical property discovery

Patent metadata

Field	Value
Publication number	US-12249148-B2
Application number	US-202217656296-A
Country	US
Kind code	B2
Filing date	Mar 24, 2022
Priority date	Mar 24, 2022
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

According to one embodiment, a method, computer system, and computer program product for identifying one or more intrinsic physical properties of one or more objects is provided. The present invention may include identifying one or more objects in a video set, extracting observable physical properties of the identified one or more objects from the video set, including one or more trajectories, and inferring, by a property-based graph neural network, intrinsic properties of the one or more objects based on the trajectories.

First claim

What is claimed is:

1. A processor-implemented method for identifying one or more intrinsic physical properties of one or more objects, the method comprising:
identifying one or more objects in a video set;
extracting observable physical properties of the identified one or more objects from the video set, including one or more static properties and one or more dynamic properties;
inferring, by a property-based graph neural network, intrinsic properties of the one or more objects based on the one or more dynamic properties;
representing the intrinsic properties of all identified objects in the video set as a graph comprising nodes and edges, wherein each node represents an identified object of the one or more identified objects and each edge represents a charge between two connected objects; and
operating an autonomous device based on the one or more inferred intrinsic properties of the one or more objects.

2. The method of claim 1 , further comprising:
linking one or more common objects of the one or more identified objects throughout the video set based on the one or more static properties.

3. The method of claim 1 , further comprising:
identifying a video as belonging to the video set based on identifying an object appearing in the video and the video set as a common object.

4. The method of claim 1 , further comprising:
aggregating the inferred intrinsic properties of the one or more objects for the videos comprising the video set to improve an inference accuracy of the property-based graph neural network.

5. The method of claim 1 , operating the autonomous device to move an object to create an interaction between at least two objects within a video of the video set.

6. A computer system for identifying one or more intrinsic physical properties of one or more objects, the computer system comprising:
one or more processors, one or more computer-readable storage medium, and program instructions stored on the one or more computer-readable storage medium for execution by at least one of the one or more processors via the one or more computer-readable storage medium, wherein the computer system is capable of performing a method comprising:
identifying one or more objects in a video set;
extracting observable physical properties of the identified one or more objects from the video set, including one or more trajectories;
inferring, by a property-based graph neural network, intrinsic properties of the one or more objects based on the trajectories;
representing the intrinsic properties of all identified objects in the video set as a graph comprising nodes and edges, wherein each node represents an identified object of the one or more identified objects and each edge represents a charge between two connected objects; and
operating an autonomous device based on the one or more inferred intrinsic properties of the one or more objects.

7. The computer system of claim 6 , wherein the method further comprises:
linking one or more common objects of the one or more identified objects throughout the video set based on the one or more static properties.

8. The computer system of claim 6 , wherein the method further comprises:
identifying a video as belonging to the video set based on identifying an object appearing in the video and the video set as a common object.

9. The computer system of claim 6 , wherein the method further comprises:
aggregating the inferred intrinsic properties of the one or more objects for the videos comprising the video set to improve an inference accuracy of the property-based graph neural network.

10. The computer system of claim 6 , wherein the method further comprises:
operating the autonomous device to move an object to create an interaction between at least two objects within a video of the video set.

11. A computer program product for identifying one or more intrinsic physical properties of one or more objects, the computer program product comprising:
one or more computer-readable storage medium and program instructions stored on at least one of the one or more computer-readable storage medium, the program instructions executable by a processor to cause the processor to perform a method comprising:
identifying one or more objects in a video set; extracting observable physical properties of the identified one or more objects from the video set, including one or more trajectories; and
inferring, by a property-based graph neural network, intrinsic properties of the one or more objects based on the trajectories;
representing the intrinsic properties of all identified objects in the video set as a graph comprising nodes and edges, wherein each node represents an identified object of the one or more identified objects and each edge represents a charge between two connected objects; and
operating an autonomous device based on the one or more inferred intrinsic properties of the one or more objects.

12. The computer program product of claim 11 , wherein the method furcher comprises:
linking one or more common objects of the one or more identified objects throughout the video set based on the one or more static properties.

13. The computer program product of claim 11 , wherein the method furcher comprises: identifying a video as belonging to the video set based on identifying an object appearing in the video and the video set as a common object.

14. The computer program product of claim 11 , wherein the method furcher comprises:
aggregating the inferred intrinsic properties of the one or more objects for the videos comprising the video set to improve an inference accuracy of the property-based graph neural network.

15. The computer program product of claim 11 , wherein the method further comprises operating the autonomous device to move an object to create an interaction between at least two objects within a video of the video set.

Assignees

• IBM

Inventors

• Chen Zhenfang
• Gan Chuang
• Wu Bo
• Wang Dakuo

Classifications

• G06V20/50

  Context or environment of the image · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/30241

  Trajectory · CPC title

• G06V10/62

  relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title